System for placing objects on a surface and method thereof

ABSTRACT

A system for placing objects on a surface. The system may include a base, a robotic arm coupled, at an end thereof, to the base, an end effector coupled to the other end of the robotic arm. The end effector may be configured for releaseably coupling to an object to be placed on the surface. The system may further include one or more sensor units on a sensor frame. The one or more sensor units may be configured for sensing a two-dimensional profile data including at least two two-dimensional profiles together comprising at least three boundary portions of the object to be placed and at least three boundary portions of objects on the surface. At least two of the three boundary portions of the object to be placed may be from substantially non-parallel sides. At least two of the three boundary portions of the objects on the surface may be from substantially non-parallel sides. The system may further include a processor configured to determine at least three degrees of freedom of the object to be placed with respect to the sensor frame and six degrees of freedom of the sensor frame with respect to the objects on the surface in a three-dimensional space for determining a current pose of the object to be placed with respect to the objects on the surface based on the two-dimensional profile data. Further, the system may be configured to place the object based on differences between the current pose and a desired pose of the object to be placed determined from a model of objects on the surface in the three-dimensional space.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of the Singapore patentapplication No. 10201608187P filed on 30 Sep. 2016, the entire contentsof which are incorporated herein by reference for all purposes.

TECHNICAL FIELD

Embodiments generally relate to a system for placing objects on asurface and a method thereof. In particular, embodiments relate to anautomated system/apparatus for placing a plurality of objects in apre-determined arrangement on a surface and a method of placing aplurality of objects in a pre-determined arrangement on a surface.

BACKGROUND

On-site construction robotics has been constantly discussed andanticipated by both academia and industry. Multiple attempts have beenmade with limited success in addressing the complexity and problems of abuilding site, especially when it is a highly unstructured site.

For example, processes of placing surface finishing elements and, inparticular, the process of laying tiles has seen several attempts atautomation. However, until this present day, there is still no workingprocess that is available in the market.

The majority of these attempts were research projects that have remainedat a conceptual stage like the TILEBOT and the SHAMIR projects. Bothapproaches use large and heavy machinery and are thus targeted towardsfloor tiling of only large surfaces, such as in retail stores, and areunable to operate in narrow, confined spaces such as in a typicalresidential floor plan. Further, this machinery may also require newsafety and security measures that are uncommon for construction sites.

On the other hand, U.S. Pat. No. 9,358,688 describes a machine foraligning items having three edge sensors for detecting an edge alignedalong an X-X edge of the first laid item and an edge aligned along a Y-Yedge of the second laid item so as to position the new item relative tothe X-X edge and the Y-Y edge of the first and second laid items by aset distance away from the respective X-X edge and the Y-Y edge.However, the machine of U.S. Pat. No. 9,358,688 is merely for localisedplacement of items with consideration of immediate neighboring laiditems without taking into account of a global view of the overall area.

SUMMARY

According to various embodiments, there is provided there is provided asystem for placing objects on a surface. The system may include a base,a robotic arm coupled, at an end thereof, to the base, an end effectorcoupled to the other end of the robotic arm. The end effector may beconfigured for releaseably coupling to an object to be placed on thesurface. The system may further include one or more sensor units on asensor frame. The one or more sensor units may be configured for sensinga two-dimensional profile data including at least two two-dimensionalprofiles together comprising at least three boundary portions of theobject to be placed and at least three boundary portions of objects onthe surface. At least two of the three boundary portions of the objectto be placed may be from substantially non-parallel sides. At least twoof the three boundary portions of the objects on the surface may be fromsubstantially non-parallel sides. The system may further include aprocessor configured to determine at least three degrees of freedom ofthe object to be placed with respect to the sensor frame and six degreesof freedom of the sensor frame with respect to the objects on thesurface in a three-dimensional space for determining a current pose ofthe object to be placed with respect to the objects on the surface basedon the two-dimensional profile data. Further, the system may beconfigured to place the object based on differences between the currentpose and a desired pose of the object to be placed determined from amodel of objects on the surface in the three-dimensional space.

According to various embodiments, there is provided a method for placingobjects on a surface. The method may include providing a system. Thesystem may include a base, a robotic arm coupled, at an end thereof, tothe base, an end effector coupled to the other end of the robotic arm.The end effector may be configured for releaseably coupling to an objectto be placed on the surface. The system may further include one or moresensor units on a sensor frame. The one or more sensor units may beconfigured for sensing a two-dimensional profile data including at leasttwo two-dimensional profiles together comprising at least three boundaryportions of the object to be placed and at least three boundary portionsof objects on the surface. At least two of the three boundary portionsof the object to be placed may be from substantially non-parallel sides.At least two of the three boundary portions of the objects on thesurface may be from substantially non-parallel sides. The system mayfurther include a processor configured to determine at least threedegrees of freedom of the object to be placed with respect to the sensorframe and six degrees of freedom of the sensor frame with respect to theobjects on the surface in a three-dimensional space for determining acurrent pose of the object to be placed with respect to the objects onthe surface based on the two-dimensional profile data. The method mayfurther include placing, using the system, the object based ondifferences between the current pose and a desired pose of the object tobe placed determined from a model of objects on the surface in thethree-dimensional space.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, like reference characters generally refer to the sameparts throughout the different views. The drawings are not necessarilyto scale, emphasis instead generally being placed upon illustrating theprinciples of the invention. In the following description, variousembodiments are described with reference to the following drawings, inwhich:

FIG. 1 shows a schematic diagram of a system for placing objects on asurface according to various embodiments;

FIG. 2A and FIG. 2B show an example of the mobile object placementstrategy;

FIG. 3A shows a kinematic diagram of an example of a robotic armaccording to various embodiments;

FIG. 3B shows a kinematic diagram of another example of a robotic armaccording to various embodiments;

FIG. 4 shows an example of a composite joint according to variousembodiments;

FIG. 5A to FIG. 5C show the end effector of the system according tovarious embodiments;

FIG. 5D shows a profile sensor and a two-dimensional profile obtained bythe profile sensor according to various embodiments;

FIG. 6A shows schematically how the sensors (and sensing planes) arearranged for rhombus shaped objects (for reconfigurable embodiment ofbasic concept);

FIG. 6B shows schematically how the sensors (and sensing planes) arearranged for hexagon shaped objects (for reconfigurable embodiment ofbasic concept);

FIG. 6C and FIG. 6D shows an example of a reconfigurable embodiment (ofaugmented sensing concept including dimensions) of an end effector forsquare objects;

FIG. 6E shows an example of a single measurement station according tovarious embodiments;

FIG. 7 shows a custom profile sensor configuration with three camerasaccording to various embodiments;

FIG. 8 shows a control diagram of an example of “eye-in-hand” sensingaccording to various embodiments;

FIG. 9A shows the basic concept with three sensors according to variousembodiments;

FIG. 9B shows the basic concept including dimensions with three sensorsaccording to various embodiments;

FIG. 9C shows the augmented concept with three sensors according tovarious embodiments;

FIG. 9D shows the augmented concept including dimensions with foursensors according to various embodiments;

FIG. 10 shows example of common tiling defects;

FIG. 11 shows an object storage apparatus and a refill trolley accordingto various embodiments;

FIG. 12 shows a bonding material application apparatus according tovarious embodiments;

FIG. 13A shows a structure for measuring dimensions of a tile accordingto various embodiments. FIG. 13B shows a rectangular tile beingmeasured. FIG. 13C shows a square tile being measured.

FIG. 14 shows a schematic top view of a system for placing objects on asurface according to various embodiments;

FIG. 15A shows a schematic top view of a system for placing objects on asurface according to various embodiments;

FIG. 15B shows a schematic side view of the system of FIG. 15A withrobotic arm extended;

FIG. 16 shows an example of the system application on prefabricatedprefinished volumetric construction (PPVC);

FIG. 17 shows an example of the system including a gantry systeminstalled in a prefabrication facility.

DETAILED DESCRIPTION

Embodiments described below in context of the apparatus are analogouslyvalid for the respective methods, and vice versa. Furthermore, it willbe understood that the embodiments described below may be combined, forexample, a part of one embodiment may be combined with a part of anotherembodiment.

It should be understood that the terms “on”, “over”, “top”, “bottom”,“down”, “side”, “back”, “left”, “right”, “front”, “lateral”, “side”,“up”, “down” etc., when used in the following description are used forconvenience and to aid understanding of relative positions ordirections, and not intended to limit the orientation of any device, orstructure or any part of any device or structure. In addition, thesingular terms “a”, “an”, and “the” include plural references unlesscontext clearly indicates otherwise. Similarly, the word “or” isintended to include “and” unless the context clearly indicatesotherwise.

Various embodiments of a system or an apparatus or a method for placingobjects on a surface have been provided to address at least some of theissues identified earlier.

Various embodiments have been provided to address problems in asub-domain of construction. Various embodiments have provided anapparatus or a system or a method that is capable of automaticallyplacing objects in an arrangement on a surface (e.g. a planar surface),particularly, automatic laying of tiles on a floor on-site, and itsusage. According to various embodiments the apparatus or the system maybe placed on-site. The planar surface in here may be referring to a flatsurface that could be horizontal, inclined or vertical, for example afloor, a wall, or a ceiling. The planar surface may be a pre-fabricatedfloor, wall, ceiling in prefabricated prefinished volumetricconstruction (PPVC) or a prefabricated bathroom unit (PBU) which may bebuilt off-site. The planar surface may also be a work bench, a countertop, a desk top, a pedestal, a work table or any suitable supportsurface.

In one embodiment, an automated apparatus or system is provided. Theautomated apparatus or system may be utilized to place objects in aparticular arrangement to cover a planar surface. The object may be anarchitectural object or non-architectural object. In one exemplaryembodiment, the architectural object may be a tile made using ceramic,porcelain, natural stone such as granite or marble, polymers, glass ornatural and/or processed wood such as timber. In another exemplaryembodiment, the architectural object may be panels. In one exemplaryembodiment, the non-architectural object may be solar panels, or otherelectronic or electrical components.

FIG. 1 shows a schematic diagram of a system (or an apparatus) 100 forplacing objects on a surface according to various embodiments. In oneembodiment, the system 100 may include an actuated mobile platform 110,a robotic arm 120 (or manipulator or articulated robot), an end effector130, one or more suction cups 140, or one or more sensor units 150, anair-compressor, a vacuum ejector, an object storage apparatus 160, abonding material application apparatus 170, and/or a control mechanism180.

In one embodiment, the system 100 may be able to fit through the doorsof residential/housing units on-site (90-80 cm) and may be able to workand manoeuvre in narrow spaces such as corridors (120-110 cm).Therefore, dimensions of the system 100 may not exceed 70 centimetres(cm) in width and 90 cm in length, while a maximum height of the system100 with the various components retracted does not exceed 180 cm. Thedimensions of the system 100 may further not exceed a length that wouldnot allow its footprint to fit inside a circle of 100 cm diameter, forthe system to be able to rotate in aforementioned corridors. Further,the weight and size of the system 100 may be configured such that it canoperate in small rooms. Accordingly, the system 100 may be able to befit in residential units.

In one embodiment, a mobile object placement strategy is developed tocorrespond or tailor to the size of the system 100. FIG. 2A and FIG. 2Bshow an example of the mobile object placement strategy. As shown, thesystem 100 may cover the whole room 201 by advancing to the nextposition after placing a patch of objects in each position until onlythe area expanding from the door opening to the opposite wall of theroom is left uncovered. Subsequently, the system 100 then makes a 90degree turn and continues placing objects and advancing backwards untilthe system 100 leaves the room through the door opening 203. Here,“patch” refers to a subassembly of objects within the reach of therobotic arm 120 of the system 100 in the current position of the base110 of the system 100. This placement strategy and dimensioning of thesystem enables the whole system 100 to work avoiding standing on coveredsurfaces. A body 102 of the system 100 may include a light and stableframe which can be lifted using a pallet jack or custom transportationapparatus if necessary under special circumstances on the constructionsite. The custom transportation apparatus may be similar to a palletjack and may be a customised structure configured for forks to slide inon the side or the other suitable portions of the customised structure.

According to various embodiments, the placement method of an object maybe based on one or more sides of the object (e.g., side surfaces of anobject geometrically extruded from a polygonal outline consisting ofstraight segments and/or the upper surface defined by such outline). Theplacement of the object may occur once the object is alignedsubstantially in parallel to one or more sides and the upper surface ofan already-placed object, and simultaneously within a close proximity tothe already-placed object. In one embodiment, the placement may reachsub-millimetre (sub-mm) accuracy. For example, with an accuracy (gapsize) of down to 0.3-0.2 mm. In one embodiment, the system 100 enablespreviously unattainable/unprecedented and/or surprisingly level ofaccuracy. Further, the system 100 may also be configured to addresscommon tiling defects as shown in FIG. 10.

According to various embodiments, the one or more sensor units 150 maybe configured to measure/sample/acquire parts of cross-sections ofrelevant objects, wherein one or multiple such cross-sections may berepresented in a profile. These cross-sections may also be referred toas side portions or edge portions or boundary portions of objects. Theparts of the cross-sections of relevant objects may include a portion ofa top surface of the object to be placed extending inwards from an edgeof the object to be placed. The parts of the cross-sections of relevantobjects may also include a portion of a top surface of the object on thesurface extending inwards from an edge of the object on the surface.Further, the one or more sensor units 150 may be configured to measureone or multiple such profile in its respective sensing planes.Accordingly, the one or more sensor units 150 may be configured to sensea two-dimensional profile of relevant objects. Hence, the sensor unitmay include an integrated profile sensor or a custom profile sensor. Theintegrated profile sensor may typically measure a single profile. Thecustom profile sensor may measure one or multiple profiles (all by meansof triangulation) and typically comprises multiple discrete components(e.g. an imaging device and light emitting devices).

The actuated mobile platform 110 may also be referred to as a base ormobile base. The actuated mobile platform 110 may be able to moveindependently along an X-axis and a Y-axis with respect to the planarsurface (i.e., having a zero non-holonomic kinematic constraints formovement in an XY plane). Furthermore, the actuated mobile platform 110may also rotate around a Z-axis in a translation-free manner, wherebythe Z-axis is a normal to the planar surface.

According to various embodiments, the actuated mobile platform 110 maybe configured to be movable by including wheels 112 or legs or wheeledlegs, wherein there may be at least one drive wheel or at least oneactuated leg. For example, the wheels 112 may include spherical wheels,or Swedish or mecanuum wheels, or castor wheels. The legs may includemultiple articulated legs. The wheels 112 may be steered wheels such asomni drives, actuated castor wheels, powered steered caster wheels,steered motor-in-wheel-drives, swerve and steer modules or pivot drives.The wheels may be suspended in a non-rigid fashion (in particular ifmore than three wheels). Accordingly, the base 110 may be able todrive/walk and navigate between the locations. At each location, thebase 110 may be rendered stationary.

According to various embodiments, the acutated mobile platform 110 mayalso include a gantry structure.

In one embodiment, the actuated mobile platform may be capable of movingalong the surface on which the objects may be placed by couplingmultiple cables to the mobile platform so as to suspend the mobileplatform above the surface, similar to a cable-driven parallel robot.The multiple cables may be spanning from the platform to respectivesupport members erected around the perimeter of the surface. Themultiple cables may then be adjustably extended and retracted in acoordinated fashion to maneuver or move the platform across variouslocations above the surface. The actuated mobile platform with the cablemechanism may be assembled for a larger room from easily portablecomponents. Accordingly, the actuated mobile platform may be withoutwheels or legs. When the objects to be placed are tiles, after theplacement of the tiles, the gaps between the tiles could also begrouted, before the bonding material has fully cured or set and it ispossible to walk on the tiled surface. The materials (e.g. tiles) maylikely be stored off-platform. In this embodiment, the pose (i.e.position and orientation in three dimensional space) of the platform isalways known (from the controlled lengths of the cables, with limitedaccuracy, which may be better than in the case of wheeled locomotionwhere based on odometry).

Further, as the platform is suspended by cables in a not entirely rigidfashion, when the manipulator mounted on the platform is placing anobject on the surface, a reaction force may act on the manipulator andbe transferred to the platform causing a resultant movement on theplatform. Accordingly, the system in this embodiment may require ahighly dynamic manipulator to compensate for the resultant movement onthe platform.

Referring back to FIG. 1, the actuated mobile platform 110 according tovarious embodiments may also include three or more electrically actuatedfeet 114 that can be extended onto the floor (as shown in FIG. 1),thereby lifting the wheels 112 off the ground, supporting the entireweight of the system 100 and rendering the system 100 stationary. In oneembodiment, a hydraulic-based system may be used to extend the feet 114onto the floor, thereby lifting the wheels 112 off the ground andrendering the actuated mobile platform 110 stationary.

In an alternative embodiment, the actuated mobile platform 110 mayinclude three or more rigid feet 114 to support the entire weight of thesystem 100. The wheels 112 may then be retracted into the platform 110,either electrically or hydraulically, and thus rendering the platform110 stationary when the platform 110 is resting with the feet 114 on theground.

According to various embodiments, the actuated mobile platform 110 mayfurther include a scissor lift or similar mechanism that may allowextension of the reach of the manipulator 120, e.g. to reach higherparts of a wall. Accordingly, the scissor lift may be coupled to thebase joint of the robotic arm 120 so as to move the robotic arm along aZ axis (at a scale of beyond a few cm) away from the actuated mobileplatform 110. Further, the platform 110 may include one, two or moretwo-dimensional or three-dimensional light detection and ranging (LiDAR)sensors mounted on the base 110 (e.g. two sensors on opposite corners)for obstacle avoidance, workspace and safety monitoring and possiblylocalization. The base 110 may also include an array of ultrasonicdistance sensors around the perimeter of the platform 110 for obstacleavoidance or workspace monitoring. Furthermore, the base 110 may includebumper switches covering the perimeter of the base 110 such that thebumper switches may open on impact with an object. The bumper switch maybe configured to trigger safety functions (as a last resort) to haltparts of the system 100. In addition, the base 110 may also includeelectrical switches configured to be opened when contact with the flooris lost. For example to stop the platform 110 from driving off a step.The electrical switches may be disposed on each side of the base 110.

According to various embodiments, the robotic arm 120 of the system 100may include multiple joints. In one exemplary embodiment, the roboticarm 120 may include six or more joints (as shown in FIG. 1) that arespaced apart. These joints may be connected through links. The linksbetween these joints may form a kinematic chain. These joints may bereferred to as a base joint 121, a shoulder joint 122, an elbow joint123, a first wrist joint 124, a second wrist joint 125 and a third wristjoint 126. The third wrist joint 126 may also be referred to as an endjoint. The base joint 121 may be coupled to the actuated mobile platform110. The end joint 126 is located farthest away from the base joint 121.

In one embodiment, one or more of these joints 121, 122, 123, 124, 125,126 within the robotic arm 120 are actuated with linear or rotaryhydraulic actuators. These hydraulic actuators may be integratedhydraulic servo actuators. These actuators may be force-controlled. Inanother embodiment, one or more of these joints 121, 122, 123, 124, 125,126 within the robotic arm 120 are actuated with electric motors,specifically harmonic drives.

FIG. 3A shows a kinematic diagram of an example of a robotic arm 320according to various embodiments. The robotic arm 320 may have similarfeatures to a Selective Compliance Assembly Robot Arm (SCARA) typerobot. As shown, the robotic arm 320 includes series of three revolutejoints 322, 323, 324 all with parallel vertical axis. According tovarious embodiments, the robotic arm 320 may be inherently rigid in theZ axis so that it can take reaction forces from pressing an object to beplaced down (e.g. robust bearings may take the load rather than thedrives). According to various embodiments, the robotic arm 320 may havelimited workspace in the Z axis. Accordingly, in order for placement ofobjects (e.g. tiles) higher up on wall, the platform 110 may need tointegrate scissor lift for adjusting the height of the robotic arm 320.According to one embodiment, the basis of the robotic arm 320 mayinclude a first set of 5 degree of freedom (DOF) serial kinematic withprismatic and revolute joints. Accordingly, the robotic arm 320 mayinclude a first prismatic joint 321 (i.e. joint 1) for large Zmovements. The robotic arm 320 may further include the three revolutejoints 322, 323, 324 (i.e. joint 2, joint 3, joint 4) for X/Y/yawmovements. According to one other embodiment, the robotic arm 320 may beconfigured to include a first set of 6 DOF by having an additionalprismatic joint before the last revolute joint 324 for small Z movements(not depicted in the diagram). Further, the robotic arm 320 may beconfigured to have further DOF. In one embodiment, the further DOFincludes a second set of 5 DOF serial and parallel kinematic withprismatic and revolute joints. Accordingly, the robotic arm 320 mayfurther include two serial revolute joints 325, 326 a (i.e. joint 5 andjoint 6) for alignment of the end effector 330 to a vertical surface.The robotic arm 320 may also include a composite joint 390 with a 3 DOFparallel kinematic. The composite joint may include three parallelprismatic joints 327, 328, 329 (i.e. joint 7, joint 8, joint 9) forroll/pitch/(z) adjustment. According to various embodiments the twoserial revolute joints 325, 326 a (i.e. joint 5 and joint 6) may not beneeded for floor placement. According to various embodiments, whenperforming vertical placement of the object, the revolute joint 325(joint 5) may be locked so that the motor does not need to supplyconstant torque because the composite joint 390 may be used for fineadjustment. In one other embodiment, the further DOF includes a secondset of 2 DOF serial kinematic with revolute joints. Accordingly, therobotic arm 320 may include revolute joint 325 (i.e. joint 5) andrevolute joint 326 b (i.e. joint 6 b) for roll/pitch adjustment (forfloor placement only). For wall placement, this embodiment must includean additional revolute joint 326 a (i.e. joint 6).

FIG. 3B shows a kinematic diagram of another example of a robotic arm420 according to various embodiments. The robotic arm 420 may be anarticulated robotic arm. As shown, the robotic arm 420 may include acommon 6 DOF serial kinematic with revolute joints 421, 422, 423, 424,425, 426 (i.e. joint 1 to joint 6). According to various embodiments,the robotic arm 420 may be configured to include a 5 DOF serialkinematic with revolute joints attached to a 3 DOF parallel kinematicwith prismatic joints. For example, the robotic arm 420 may includerevolute joints 421, 422, 423, 424, 426 (i.e. joint 1 to joint 4 andjoint 6) and three parallel prismatic joints 427, 428, 429 (i.e. joint 7to joint 9) without the fifth revolute joint 425 (i.e. joint 5). Thethree parallel prismatic joints 427, 428, 429 may form a composite joint490 having 3 DOF. Each of the three parallel prismatic joints 427, 428,429 may have a passive and non-actuated joint at each end. According tovarious embodiments, the fifth revolute joint 425 (i.e. joint 5) maystill be needed for placement on wall. According to various embodiments,revolute joints 421, 423, 424 (i.e. joint 2 to joint 4) could bemotorized with linear hydraulic actuators.

FIG. 4 shows an example of a composite joint 590 with a 3 DOF parallelkinematic for roll, pitch and Z axis fine adjustment. The compositejoint 590 may be configured to be coupled to an end joint 126 of therobotic arm 120 or may be part of the end joint 126 of the robotic arm120. Accordingly, the end effector 130 may be coupled to the compositejoint 590. Hence, the composite joint 590 may couple the end-effector130 to the robotic arm 120. In one embodiment, the one or more sensorunits 150 may be coupled directly to the end-effector 130. In anotherembodiment, the one or more sensor units 150 may be coupled to one endof the composite joint 590 and the end-effector 130 may be coupled toanother end of the composite joint 590. In such an embodiment, thecomposite joint 590 may be considered as part of the end-effector 130.

As shown in FIG. 4, the composite joint 590 may include at least threeparallel linear actuators 527, 528, 529 (e.g. DC stepper motor captiveleadscrew actuator or hydraulic linear servo actuator or electric linearservo actuator). According to various embodiments, the composite joint590 may be able to apply a high press force (limited by the reactionforce the rest of the kinematic is able to handle). In one otherembodiment, a composite joint may include at least two linear actuatorswith a parallel ball joint. In another embodiment, the composite joint590 may substitute the linear actuators with rotary actuators withlevers. In yet another embodiment, the composite joint 590 may also beconfigured to have Hexapod/Steward platform configuration (six parallelprismatic, joints with the revolute joints replaced with universaljoints). As shown in FIG. 4, each of the linear actuators 527, 528, 529has a revolute joint 591 at the top end and a spherical joint 592 at thebottom end. In another embodiment, the revolute joint 591 may be at thebottom end and the spherical joint 592 may be at the top end. Referringback to FIG. 4, each of the linear actuators 527, 528, 529 may becoupled to a first plate 593 via the revolute joint 591. Each of thelinear actuators 527, 528, 529 may also be coupled to a second plate 594via the spherical joint 592. The one or more suction cups 140 may becoupled to the second plate 594. Each suction cup 140 may be coupled toa vacuum source via a pipe 595. Hence, each pipe 595 may couple onesuction cup 140 to the vacuum source. Further, the second plate 594 mayinclude projections 596 or collars (around the suction cups). Accordingto various embodiments, the object 101 may be pulled against theprojections 596 or collars by suction cups. The projections 596 mayrestrict the pose of the object 101 in roll and pitch and Z position.According to various embodiments, the projections or collars may also beincluded in end effector 130 of other embodiments without parallelkinematic composite joint 590. As shown, the composite joint 590 may becoupled to the end joint 126 or end link of the robotic arm 120.According to various embodiments, a vibrator (e.g. a motor with anunbalanced mass on its driveshaft) may be coupled to the first plate593. The vibrator may not be coupled to the first plate 593 in a rigidfashion in order to prevent the transmission of vibrations. In oneembodiment, an additional linear actuator (e.g. pneumatic or hydraulicor leadscrew actuator etc.) may be configured for releasing/retractingthe vibrator independently from end effector movement. In anotherembodiment, the vibrator may be suspended or hanged on the compositejoint 590 without being coupled to any actuator (thereby allowing forcertain x/y/z movement, restricted by guides to limit lateral movementto prevent contact with other components). In this embodiment, thevibrator may be lowered onto the objet by retracting the lower platewith the three linear actuators while simultaneously moving the wholecomposite joint down. According to the various embodiments, vibrationsmay be decoupled from the end effector 130. According to variousembodiments, the vibrator may be lowered and engaged for final z-axismovement of the object 101 and/or after other suction cups have beenlifted off. The vibrator may also include a suction cup on the bottomthereof to attach to the object 101. According to various embodiments,the vibrator may also be included in other embodiments of the endeffector 130 without the parallel kinematic composite joint 590.

FIG. 5A to FIG. 5C show the end effector 130 of the system 100 accordingto various embodiments. As shown, one 132 of the surfaces of the endeffector 130 may be secured to the end joint 126. In one embodiment, theend effector 130 is detachable from the end joint 126 if needed.

The other surfaces of the end effector 130 may be coupled to one or morestructures 134. These structures 134 may be adjustable or configurablesuch that some of the side surfaces of the end effector 130 may beapproximately aligned in parallel with sides of the gripped object101(i.e., an object to be placed). FIG. 6A to FIG. 6B show examples ofother configurations of end effector 730, 731 according to variousembodiments. As shown, for example, in FIG. 6A to FIG. 6B, the sides ofthe object-to-be-placed may become approximately perpendicular to asensing plane 751 of the profile sensors 750 or sensor units mounted onthe side surfaces.

FIG. 6A shows an example of another configuration of an end effector 830according to various embodiments. As shown, a middle line formed betweentwo adjacent sides (with an angle that is approximately between 60degrees and 120 degrees) of the gripped object may become alignedapproximately perpendicularly to the sensing plane 851 of a profilesensor 850 or sensor unit mounted on one of the side surfaces of the endeffector.

In one embodiment, and as shown in 5A to FIG. 5C, the suction cups 140mounted on the lower surfaces may be distributed within the area of theupper surface of the gripped object 101. According to an embodiment, thesuction cups 140 may be symmetrically distributed.

According to various embodiments, the end effector 130, 730, 830 mayinclude one or more two dimensional (2D) or three dimensional (3D) LiDARsensors, an inclinometer (or an inertial measurement unit or an attitudereference system or an attitude heading reference system or anorientation reference system; all later referred to as the latter), aforce-torque sensor (typically referred to as F/T sensor, typicallyproviding measurements in 6 axes, in particular of use for placinginterlocking objects such as timber or vinyl tiles), a pneumatic valveand a vacuum ejector, a precision rabbet with a calibrated geometry thatcan be used for measuring dimensions of the current object, featureswhich provide a solid contact surface to the object's upper side toconstrain its position in negative z-dimension, as well its orientationin roll and pitch.

Referring back to FIG. 1, the system 100 also includes one or moresuction cups 140 mounted on the lower surface of the end effector 130.These suction cups 140 are coupled to a vacuum source. When the vacuumsource is switched-on, the suction cups 140 adhere to an upper surfaceof the object 101 to be placed (such as a tile).

In the embodiment shown in FIG. 5A to FIG. 5C, the structures 134 arespatially separated and may to some degree mechanically isolate thelower surface(s) from the structure(s) 134 connecting the side surfaces.Such arrangement may help to reduce the stress on the structures 134connecting the side surfaces and may redirect the stress to the endjoint 126 of the robotic arm 120. Hence, any deformations (elastic orplastic) that may arise may not affect the sensor poses.

In one embodiment, the vacuum source for the suction cups 140 may be anair compressor in conjunction with a venturi vacuum generator (alsoknown as a vacuum ejector). In an alternative embodiment, the vacuumsource may be a vacuum pump or a vacuum blower, especially in the casewhere the system is optimized to handle objects with very roughsurfaces.

Furthermore, the connections of the suction cups 140 to a vacuum sourceare switchable individually or in groups using mechanical means so as toadapt to different shaped and sized objects. Alternatively, theconnections of the suction cups 140 may be switchable usingelectro-mechanical means and subsequently be actuated and controlled.

As shown in FIG. 5A to FIG. 5C, the system 100 may also include multipleprofile sensors 150 or sensor units that are mounted on the sidesurfaces (in a way so that the surface normal vectors will lie in therespective planes in which the profiles are measured). The sensors 150are oriented in a way so that the profiles measured are samples of across-section of the object to be placed 101 (e.g., at an approximatingangle of 90 degrees or 45 degrees with respect to the object's sideplane) and at least one object on the surface 103 (with eachcross-section covering one or two edges or sides of an object) once thelatter be close enough to be within the sensor's field of view. In oneembodiment, the system 100 may include three or more profile sensors150.

It should be appreciated that the profile sensors 150 differ from edgesensors. An edge sensor outputs only a gap size (a one dimensionalmeasurement) or an edge offset from a centre line (a one dimensionalmeasurement) or an edge point. This information would (a) not besuitable to compute a relatively good enough estimate of the pose of anobject in the presence of a chamfer or bevel around its upper surface asthe obtained point will not lie in the surface plane of the object andtherefore, would likely require additional sensors, and (b) not besuitable to compute an estimate of the floor plane or other geometricfeatures.

The profile sensors 150, as shown in FIG. 5D, on the other hand, provideaccess to two-dimensional coordinates of points in the measurement plane157 (which constitute a “profile” 159), with 500 points or moretypically, spread over a distance of 25 mm or more, yielding a samplingresolution in that direction of 0.2 mm or higher. FIG. 5D shows theprofile sensor 150 and the two-dimensional profile 159 obtained by theprofile sensor according to various embodiments.

A profile may be then be analysed by a custom algorithm implementedthrough a software program (i.e., based on a particular code) runningeither on an embedded processor within the sensor or on a processorwithin a control computer.

The analysis may include steps of fitting lines or more complexgeometric shapes to a subset of profile points, projecting selectedpoints onto such shapes etc. In one embodiment, the results may bereferred to as profile features.

A combination of such profile features from multiple profile sensors 150may be transformed to a common 3D coordinate system. These profilefeatures are then used to compute more abstract properties of the object101 and potentially other entities (e.g., objects previously placed 103on the planar surface, physical references, of the planar surfaceitself). This enables computing up to 6 degrees of freedom (DOF) oftheir pose or some of their dimensions.

In one embodiment, measurements obtained from two profile sensors 150are needed to align a side of the object 101 (measuring in planes eitherapproximately perpendicular or at a 45 degrees angle to the respectiveside, spaced apart at the largest feasible distance) with a side of apreviously placed object 103 in 3D space, in order to estimate a highenough number of the degrees of freedom of the previously placed object103 (e.g., to estimate the orientation of its top surface, or the yawcomponent of its pose, which is not fully possible with single sensor).

The measurements from the profile sensors 150 are synchronously takenfrom a single pose of the end effector 130 (no “multiple views” arerequired) and combined to compute the currently to be placed object 101or previously placed object 103 properties, in order to be used forclosed-loop control of the end effector 130. In one embodiment, themeasurements are taken at a rate of at least 30 hertz (Hz). In addition,using only measurements from a single pose ensures that the objectproperties are free of errors introduced by inaccuracies in thetransformations between subsequent poses of the end-effector 130.

The end effector 130 may be configured in a way so that each side of thecurrent object 101 that is to be aligned to at least one side of apreviously placed object 103 is perceived by at least two profilesensors 150. For a non-rectangular outline such as from a triangular,rhombic or hexagonal shaped object as shown in FIG. 6A and FIG. 6B, morethan two profile sensors 750 may be needed (if more than one side is tobe aligned).

In one embodiment, a hexagonal shaped outline may require at least threeprofile sensors 750 (if three sides are to be considered for determiningthe goal pose), wherein there is one profile sensor on each side.

According to various embodiments, feasible configurations of the sensingconcept may include the following.

The feasible configuration may include a basic sensing concept. Thebasic sensing concept may involve continuously measuring the full 6 DOF(or 3 DOF if z/pitch/roll are mechanically restricted) of the currentlygripped object 101 and 6 DOF of previously placed objects 103 as shownin the examples of FIG. 5A, FIG. 6A and

FIG. 6B. For example, it may be sufficient to measure 5 DOF of a firstobject on the surface and just 1 DOF of a second object on the surfaceif information from a model according to various embodiments is used inaddition. FIG. 5A shows a non-reconfigurable (fixed) embodiment of thebasic concept with 4 sensors (for rectangular objects of limitedspecific sizes, e.g. 300×300 mm, 300×600 mm). FIG. 6A showsschematically how the sensors (and sensing planes) are arranged forrhombus shaped objects (for reconfigurable embodiment of basic concept).FIG. 6B shows how the sensors (and sensing planes) are arranged forhexagon shaped objects (for reconfigurable embodiment of basic concept).FIG. 9A shows the basic concept with 3 sensors.

The feasible configuration may also include an augmented sensingconcept. The augmented sensing concept may involve measuring anadditional DOF for previously placed adjacent objects 103 (thus reachingfull 6 DOF) by sampling an additional side of these objects that isadjacent to the side the current object 101 is to be aligned to. Thismay reduce reliance on estimated properties from the model and increaseplacement precision. However, this may in return increase the size ofthe end-effector as shown in the examples of FIG. 5B and FIG. 5C. FIG.5B shows a non-reconfigurable (fixed) embodiment of the augmentedconcept with 5 sensors (for rectangular objects of limited specificsizes, e.g. 300×300 mm, 300×600 mm). FIG. 5C shows a non-reconfigurable(fixed) embodiment of the augmented concept with 4 sensors (forrectangular objects of limited specific sizes, e.g. 300×300 mm, 300×600mm). FIG. 9C shows the augmented concept with 3 sensors.

The feasible configuration may also include a basic/augmented sensingconcept including dimensions. The basic/augmented sensing conceptincluding dimensions may involve additionally allowing measuring of oneor more dimensions of the current, gripped object by sampling anadditional side of it which is opposite to a side that is to be aligned,without requiring an auxiliary apparatus. FIG. 6C and FIG. 6D show areconfigurable embodiment of the augmented concept including dimensionswith four sensors. FIG. 9B shows the basic concept including dimensionswith 3 sensors. FIG. 9D shows the augmented concept including dimensionswith 4 sensors.

Profile sensor 150, 750, 850 or the sensor unit as described herein mayinclude an integrated profile sensor or a custom (or discrete) profilesensor. An “integrated” profile sensor typically integrates image sensor(typically complementary-metal-oxide-semiconductor: CMOS orcharge-coupled-device: CCD), lens, light source (typically laser diode),pattern generator (typically diffractive optical element to generateline) and processing unit (typically field programmable gate array:FPGA) into a single enclosure that is configured for use in industrialautomation. The integrated profile sensor is typicallyfactory-calibrated (e.g. intrinsic parameters of lens, pose of laserplane with respect to sensor etc.) and is able to provide metric 2Dmeasurement points from the sensing plane (by means of triangulation) orderived, more abstract features with sub-mm resolution and at rateshigher than 100 Hz. The integrated profile sensor typically onlymeasures in a single plane (so there is usually one laser line projectedonto the object surfaces).

According to various embodiments, the integrated profile sensor may becoupled to the end effector 130 in a non-reconfigurable configuration.Accordingly, the integrated profile sensor may be coupled to the fixedstructure 134 as shown in FIG. 5A to FIG. 5C. Accordingly, the endeffector 130 may have little flexibility for accommodating objects ofdifferent sizes.

According to various embodiments, as shown in FIG. 6C and FIG. 6D, theintegrated profile sensor may be coupled to the end effector 830 in areconfigurable configuration. For example, the structure to which theintegrated profile sensors are attached to may be reconfigurable forsquare objects using prismatic joints. E.g. for 20×20 to 60×60 cmobjects. Accordingly, the integrated profile sensors 850 may be mountedon extendible arms (e.g. comprising a telescopic rail, a leadscrew and astepper motor) or the carriage 852 of a linear axis 853 (e.g. steppermotor actuated). Each arm may have 1 DOF and may not rotate. This mayallow precise re-positioning of the sensors, e.g. realizing the sensingconcept of FIG. 9D with 3 arms arranged at 90 degree angles. In anotherexample, the structure to which the integrated profile sensors areattached to may be reconfigurable for rectangular objects usingprismatic joints, e.g. for 60×30 cm objects. Similarly, the structuremay include extendible arms or linear axes with carriages. Further, thesensor arms can pivot or rotate around the axis of the end joint (i.e.wrist 3) of the manipulator. The extendible arms may be attached to thepart of the end effector coupled to the manipulator end joint via rotaryactuator. According to another embodiment, pairs of arms may beconnected via extending linear actuators. Accordingly, the two armsforming the ends of the chain need to be “grounded” to the end joint(i.e. wrist 3) via actuators (so 7 linear actuators may be required for3 arms) to prevent rotation of the whole chain around the end jointaxis. According to various embodiments, another rotary actuator at theend of the arm may be needed to set the optimal angle of the sensingplane for non-square objects. In a further example, the structures towhich the integrated profile sensors are attached to may bereconfigurable using revolute joints, e.g. resulting in a 3 DOF serialkinematic (as in an articulated robot) for each sensor. Joint 1 mayshare axis with end joint (i.e. wrist 3) of the manipulator. The gripperstructure underneath may be connected via thru-bore of joint 1 to thestructure coupling the end effector to the end joint of the manipulator.Joint 2 and Joint 3 may have parallel offset axis. The sensor assemblyon joint 3 can be positioned with 3 DOF in a plane above the object.Measurement of profiles in more than one location (i.e. with multiple3DOF serial kinematics) may need revolute joints with high encoderresolution and stiff links to determine the resulting sensor posesaccurately (for transforming profile data into common frame withoutintroducing significant errors). This example may be more suitable orfeasible with single custom profile sensor as described below.

In a “custom” (or “discrete”) profile sensor, an imaging device (andusually lens) and light projecting component(s) are separate componentsrather than comprising a factory-precalibrated unit. In the customprofile sensor, measurements in multiple planes are possible (e.g.resulting in projections of two lines or a stripe pattern; subject tohow the sensor data is processed). The components and theirconfiguration (including the geometry of the setup) can also becarefully chosen in order to achieve the targeted measurementcharacteristics. According to various embodiments all parts of thecustom profile sensor may be calibrated in an elaborate procedure inorder to approach the maximum theoretical measurement performance.Further, the calculation of metric profile points in a defined sensorreference frame using the calibrated model is delegated to a processorwhich integrates the components. The processor may be providedseparately. According to various embodiments, such a setup may allow fora larger field of view and greater flexibility over integrated sensors,but substantial effort may be needed to achieve stability androbustness, especially under harsh conditions. In one embodiment, oneclass of imaging devices which is widely available and outputs full areaimage data may be used. With a high-resolution sensor and the highmeasurement rates required for control (at least 20 Hz, ideally 100 Hzor more), this may lead to high data rates to the main processing unit,particularly with multiple sensors. Accordingly, these may push thelimits of both the sensor interface (bandwidth) and the computationalload the processing unit is able to handle. In another embodiment,another class of imaging devices that is sparsely available at the timeof writing may output the results of a preprocessing stage (which aremore abstract and much smaller in byte size). Typically, an FPGAdeterministically (and at high speeds) executes preprocessing operationson the image data, thus enabling use of the results for real-timeapplication behaviour (such as typically required for a closed-loop orfeedback control system). The central preprocessing step would be theextraction of the light pattern projected onto the object (typicallylaser line(s)) from the image data (e.g. sub-pixel accurate peakdetection in each sensor array column). Due to the drastically reduceddata size per frame, much higher frame rates can be transmitted thanwith conventional high-resolution industrial cameras (providing aresolution of 10M pixel or more). The remaining processing steps inobtaining metric data (e.g. undistortion, transformation into sensorreference frame, intersection of rays with laser plane etc.) still needto be executed by the processor which integrates the components. Theprocessor may be provided separately.

Various embodiments of the custom profile sensors seek to providemeasurements of cross-sections at sufficient resolution for rectangularobjects with sides of 20 cm or more length (such as 20×20, 30×30, 60×30,50×50 or 60×60 cm).

According to various embodiments, the custom profile sensors may becoupled to the end effector 130 in a non-reconfigurable configuration.For example, the end effector may include one custom profile sensor witha single camera looking “over a corner” of the rectangular object to beplaced such that the two adjacent corners are still (or almost) withinthe field of view of the camera. The angle between the optical axis ofthe camera and the normal of the object surface may be larger than zero(i.e. the camera is “not straight down” looking). The custom profilesensor may include laser lines projected onto the objects (e.g. as inFIG. 7). The laser lines may be substantially parallel to the x axis ofimaging plane of the camera. The angle between y axis of imaging planeof the camera and the normal of the laser plane may be sufficientlylarge to result in the required z axis resolution (with respect to theobject frame). This configuration may be implemented with a suitableimaging sensor and lens with sufficient resolutions.

According to various embodiments, the number of custom profile sensorsemployed depends on the available camera (and lens) resolution and the(range of) size(s) of objects to be measured.

In another example, the end effector may include two custom profilesensors with two cameras looking “over sides” (2 adjacent sides) of therectangular object to be placed such that profiles measured near bothcorners of one side yield sufficient lateral resolution. The camera maybe looking straight down in the middle of a side or from a corner at anangle. The laser lines may be projected appropriately (so that theprojection angles yield required z resolution etc.), approximatelyperpendicular to the sides. This configuration may be implemented with asuitable imaging sensor and lens with sufficient resolutions.

In yet another example, the end effector may include three customprofile sensors with three cameras looking “over corners” (3 adjacentcorners) of the rectangular object to be placed. FIG. 7 shows a customprofile sensor configuration 770 with three cameras 772 according tovarious embodiments (part of the end effector that couples with theobject to be placed not shown). As shown in FIG. 7, according to variousembodiments, an augmented sensing concept including dimensions may beimplemented with three custom profile sensors. According to variousembodiments, the three custom profile sensors may be coupled to a fixedor a non-reconfigurable frame. According to various embodiments, thethree custom profile sensors arrangement may be configured for objectsof 20×20, 30×30, 60×30 and 60'60 sizes. According to variousembodiments, only the appropriate lasers 774 may be switched on for suchan object (in FIG. 7 the field of view for all cameras and all laserplanes are depicted to show the arrangement for the different sizes).According to various embodiments, for a 60×30 configuration, either theleft or right laser projector may be switched on on each arm, dependingon the orientation in which the object is coupled. According to variousembodiments, the mounting structure may be a light and stiff carbonfibre structure.

In a further example, the end effector may include four or more customprofile sensors with four or more cameras and with appropriate laserpatterns projected.

According to various embodiments, the custom profile sensors may becoupled to the end effector 130 in a reconfigurable configuration. Thereconfigurable configuration of the custom profile sensors may besimilar in principle to that of the integrated profile sensors. However,the larger field of view should allow for smaller reconfigurationranges. Further, the profiles can be measured under optically favourablegeometries (e.g. area of lens with higher resolution used).

According to various embodiments, various sensing strategies may beprovided. According to the various embodiments, the sensing strategy mayinclude sensing on the end effector 130 of the manipulator (i.e.“eye-in-hand” sensing), which is the main strategy adopted for thevarious embodiments as described herein. FIG. 8 shows a control diagramof an example of “eye-in-hand” sensing. In this strategy, the relativepose error of the object to be placed can be obtained in the endeffector frame (and used for so called dynamic look-and-move visualservoing).

According to various other embodiments, the sensing strategy may includesensing of the objects from a fixed frame or a moving frame (i.e.eye-to-hand sensing). In one embodiment, sensing may be from a movinglink of the manipulator kinematic. However, if sensing is after joint 6of the robotic arm 420 in FIG. 3B, it is still considered moreeye-in-hand.

In another embodiment, sensing may be from the mobile base. To sense theobject poses accurately enough from a larger distance and within a widefield of view may require suitable sensing technology. Further, assensed poses transformed to the end effector frame may be influenced byerrors in the model of the manipulator kinematic, the pose of endeffector with respect to the base may have to be measured directlywithout joint data (e.g. by optical means, as using joint data with themodel may introduce the errors).

In one other embodiment, sensing may be from a separate sensingkinematic attached to the mobile base. The profile sensors may need tobe brought close enough to the area of interest. For example, the basemay include a serial kinematic (suitable for floor measurements only)comprised of one horizontal linear axis (prismatic joint) followed bythree vertical rotational axes (revolute joints). The pose of the sensorassembly with respect to the end effector may be measured directly as in[0070]. Alternatively, the pose with respect to the mobile base may needto be determined with a motion tracking system (e.g. 2D fiducial, IRreflective or active marker based) on the base. The motion trackingsystem may not be needed if repeatability of sensing kinematic veryhigh. For example, when the robotic arm has revolute joints with highencoder resolution and stiff links to determine the resulting sensorposes accurately. This example may be more suitable or feasible for asingle custom profile sensor as described below.

In yet another embodiment, sensing may be from one or multiple miniunmanned aerial vehicles (UAVs), such as drones, dispatched from themobile platform. The UAVs could land on or beside previously placedobjects in case of installation on a horizontal surface. The UAVs couldroughly position themselves relatively using vision (e.g. with help offiducials on end effector). The UAVs could be configured to sense theirpose with respect to a previously placed object fully (one such objectis sufficiently within their sensors' field of view). Their pose withrespect to the mobile base could also be determined with a trackingsystem (e.g. an LED marker based 3D motion capture system). The featuresmeasured by the UAVs could thus be transformed into a common, fixedframe to determine the full DOF of the involved objects. The UAVs couldalso accurately determine their pose with respect to the end effectorusing vision (camera on UAV and markers on end effector or vice versa),taking advantage of a smaller required field of view and allowing thecommon frame to be the end effector frame itself.

In a further embodiment, sensing may be from one or multiple measurementstations placed with the manipulator on or beside or around previouslyplaced objects (eye-to-hand sensing). Considerations may be similar tothose concerning the UAVs.

FIG. 6E shows a schematic diagram of a measurement station according tovarious embodiments. FIG. 6E shows a frame 1, a prismatic joint 2allowing re-configuration (e.g. telescopic mechanism actuated withlinear actuator), extended foot 3, retracted foot 4, cables 5 connectingto the base, Centre of Gravity 6 of whole structure, integrated profilesensor 7, sensing plane 8, a camera 9 looking at end effector, aninclinometer 10, an object 11 on surface, an object 12 to be placed onsurface, a representing end joint 13, i.e. its position during finalplacement of an object.

For example, the single measurement station (as shown in FIG. 6E) couldalso be a reconfigurable frame 1 placed around the area of the surfacewhere an object 12 is to be placed next. The structure may mountintegrated profile sensors 7 so as to implement an augmented sensingconcept including dimensions (similar to FIG. 6C and FIG. 6D). It mayinclude four retractable feet of which three are deployed at a time (1-3for left to right placement, 2-4 for right to left placement) to supportthe structure on the surface (i.e. the floor) and suspend it aboveobjects on the surface. The structure is configured so that its Centerof Gravity 6 allows it to rest stably in both feet configurations. Likevarious end effector embodiments, this frame could further comprise aninclinometer 10 (to sense an absolute reference for roll and pitch) andone or more cameras 9 arranged to capture markers on the end effectorfor precisely determining its relative pose (in case of one or morecustom profile sensors used, these may also fulfill the function of suchcameras). It may further include features which facilitate coupling tothe end effector for placement with the manipulator. This may also allowthe robotic arm to handle heavier objects as sensing structure andobject do not need to be carried at the same time. Such measurementstation may be primarily suitable for floor placement.

Accordingly, the robotic arm may pick up the measurement station fromthe base. The structure may reconfigure itself (in dimensions, feetaccording to placement direction) possibly in air, for a new object size(or if stored in smallest configuration on the base). The structure maybe placed on floor. The robotic arm may pick up an object (which may notbe strongly misaligned) and then lower it into the frame and move itlaterally into the field of view of the sensors. Placement then occurssimilar to various other embodiments. After release of the object andfinal measurement of its pose, the frame 1 is either moved to the nextlocation on the surface or brought back to the base.

According to various embodiments, if there are errors in the alignmentof the previously placed objects 103, using them as a reference (even ifsensed completely and with a high accuracy) in combination with idealplacement rules (possibly only represented as an absolute, ideal pose)would not exactly define an ideal relative goal pose for the currentobject 101. So an approximate solution may be needed, with a possibleoptimization procedure observing visual criteria and minimizing theaccumulation of global errors.

In one embodiment, the placement and positioning method may conceptuallybe subdivided into three mutually dependent but separate steps, wherebythe addressed precision depends on the requirements of each step.

The first step takes care of the navigation and positioning of thesystem 100 at the macro scale which is represented by the whole area tobe covered with objects (e.g. a room to be tiled). At this level, forinstance, the system 100 recognizes the walls of the room as theboundary using data acquired from the integrated mid-range distancesensor(s) (e.g. LiDARs) and possibly cameras. Thus the system 100 canlocalize itself with respect to the uploaded floor plan but alsogenerate a new floor plan (or map), compare it to the uploaded floorplan and alert the operator in case of significant differences, betweendigital blueprint and on-site reality. The object layout contained inthe work plan loaded into the control mechanism of the system 100 (amongthe laying sequence, mobile platform poses, the ideal floor plan andother information) is then transformed into this floor plan.

The second step is performed once the system 100 has reached the plannedworking position with an accuracy of approximately +/−10 cm or better.The robotic arm 120 scans the surface at the location where the lastplaced objects 103 are expected to be with short-range sensors (e.g.profile sensors) mounted on the end effector 130. At implementationlevel, the sensor data is used to calculate the 6 DOF pose of a known,previously laid object 103 in the end effector frame and this result maysubsequently be fused with all previously acquired information,improving (in terms of absolute accuracy and uncertainty) the currentestimate of the end effector 130 or mobile platform 110 pose in theworld reference frame. This essentially allows recalibrating the pose ofthe end effector 130 (and thus the mobile platform 110) with an accuracyof approximately +/−1 mm or better with respect to the set of objectslaid previously.

The final step is the placement process of the object 101. In order toconsistently meet the placement quality standards over largerextensions, the targeted accuracy at this stage is in the scale ofapproximately +/−0.2 mm or better. If this accuracy is beyond thecapabilities of the robotic arm 120 (e.g., due to modelling orcontroller errors) and the mechanism behaves as a dynamic system at thisresolution (e.g. bending of the robotic arm 120 due to the forcesoriginating from the friction between object and bonding material) thislast step can only be achieved by (typically real-time) closed-loop orfeedback control based on sensing of the objects.

At each iteration of the feedback loop, the control input is derivedfrom the difference between the current and the desired/goal pose of thegripped object 101.

As the object 101 is typically not to be placed on a solid surface buton a viscous material (e.g. in case of laying tiles; or the work pieceitself—object 101 with bonding material—could be considered deformable),this placement task is not naturally constrained in any of the 6 DOF(though considerable contact forces occur in some DOF). Thus the taskbecomes determining the optimal 3D pose (position and orientation) ofthe work piece relative to other 3D objects. Those can be the adjacentobjects 103, the local plane of the planar surface (i.e. the floor)and/or the reference placed by the operator. In cases where this is notconstraining enough, the ideal, pre-planned pose on the surface isfactored in. Most of the relevant properties of these objects can becalculated at each loop iteration from the measurements of high-speedand high-resolution profile sensors. Otherwise information from a modelwith random variable parameters may be used. The algorithm to determinethe optimal pose considers criteria such as distance and parallelism ofadjacent opposite sides, levelling of adjacent planes or offset from thesurface plane or potentially an absolute reference plane. The absolutereference plane may be a projected laser reference plane from a cornerof the room that is sensed from the end effector. In order to adaptivelycompensate for previous errors and bound their accumulation, a smallcorrection may be applied in addition. Using information from theaforementioned model, the object neighbourhood may be analysed for theerroneous trends or defects illustrated in FIG. 10 to determine thiscorrection in an optimization process, always trying not to exceed thethreshold of what a human eye can identify as an imprecision. Afterrelease of the object, its pose is measured again and used to update themodel.

The system 100 may also include an object storage apparatus 160 as shownin FIG. 1 and as shown in FIG. 11. FIG. 11 shows the object storageapparatus 160 and a refill trolley 162 according to various embodiments.The object storage apparatus 160 may be a structure that allows theobjects to be stacked as a pile (without any precise alignment betweenthe stacked objects) and facilitates automatic unloading of an alreadystacked pile onto it by allowing a refill trolley 162 with a fork 164that carries the pile to be inserted.

The system 100 may further include a bonding material applicationapparatus 170 for application of bonding material to the object as shownin FIG. 1 and as shown in FIG. 12.

In one embodiment, a control mechanism 180 or a control network may beneeded to operate the system 100. The information of the actuated mobileplatform 110 (incl. stabilisation mechanism), the robotic arm 120, thevacuum source, the pneumatic valves, the bonding material applicationapparatus 170 and the sensors may be fed to the control mechanism. Thisinformation may be distinguishable from the information needed for theclosed-loop control received from the end-effector 130 mounted sensors(in particular the profile sensors 150) and possibly the robotic arm 120(and the information from the model). The latter information may beutilized to control the poses (and possibly twist and wrench) of the endeffector 130 by computing the current errors from the desired pose. Thisinformation derived from the aforementioned errors may ultimately beused as control inputs for the embedded controller of the robotic arm120. The control mechanism 180 may be powered by a power source.

In one embodiment, the control mechanism 180 or the control network mayinclude a main controller for controlling high level machine logic suchas the interaction between the various components of the system 100. Thecontrol mechanism 180 or the control network may further include anembedded controller in each of the components, for example at therobotic arm 120, the mobile base 110, or the bonding materialapplication apparatus 170, for localise control and operation of therespective components.

The system 100 may also include additional structures 185 or a jig thatare specifically configured and are of precisely known geometry as shownin FIG. 13A. FIG. 13A shows the structure 185 for measuring dimensionsof a tile according to various embodiments. FIG. 13B shows a rectangulartile being measured. FIG. 13C shows a square tile being measured. Thesestructures 185 may enable measurement of the dimensions of the currentlygripped object 101. It includes features 187 to accommodate an object101 (which serve as a measurement reference, e.g., contact surfaces forthe object's underside and two adjacent sides) and auxiliary referencefeatures 189 (e.g., in a close proximity to the opposite adjacent sides)to be perceived together with the object sides (that are not in contactwith the apparatus) by the profile sensors 150 on the end effector 130.

In one embodiment, the structure 185 is configured for three differentobject sizes (that may work with an end effector 130). Possibly, thestructure 185 is mounted in such a way that the gravity force vector istilted towards an intersection point of the planes of the object thatare in contact with the structure 185.

For a cuboid shaped object, such as a rectangular tile, the robotic arm120 presses its underside and two adjacent sides against the measurementreferences 187 of the structure 185 to ensure close contact afterplacing it in the structure 185. If the object sides are not perfectlyplanar (e.g., displaying a small draught) and as a consequence the edges(which the sides form with the large upper surface of the object) arenot in contact with the measurement reference surfaces 187 of thestructure 185, the gap can be taken into account by measuring the widthwith the profile sensors after placing an object in the structure 185.

By synchronously taking a measurement with all profile sensors 150 froma single pose of the end effector 130, the distance from one side of theobject to the auxiliary reference feature 189 can be determined. As thedistance between the latter and the opposite contact surface 187 iscalibrated, and as the opposite object side is considered identical tosaid surface, the distance between the opposite object sides thatconstitutes the respective object dimension can be inferred.

As only data acquired from a single end effector pose is required andprocessed, the resulting dimensions are free from errors that a movementof the articulated arm 120 would introduce (e.g., due to inaccuracies inits kinematic model).

This concept can be adapted to objects with an outline that is notcomposed of opposite parallel lines. It results in a fully passivemeasurement structure 185 without additional sensors or mechanisms(which would add more complexity and failure modes). Without thismeasurement procedure, unobserved dimensional tolerances of the objectsof e.g. +/−0.3 mm would cause placement errors which would propagate andpotentially accumulate to erroneous offsets above acceptable thresholds.

In another embodiment, a measurement structure may differ from themeasurement structure 185 in FIG. 13A in that measurement happens beforethe object is gripped, when the object is still on stack. Themeasurement structure may include only auxiliary reference features(i.e. no contact surfaces). For example, the auxiliary referencefeatures may be in the form of a “ring” or a closed loop rim/borderstructure around the upperside of the top object on the stack. Further,either the ring is automatically moved down or the stack is moved upafter the top object has been removed. As there are no opposite contactsurfaces, after the first measurement, the end effector 130 is turned by180 degrees (using only wrist 3 joint) for a second measurement of thedistances to the opposite auxiliary reference features. Furthermore, theerror in the relative end effector pose introduced by this limitedmovement is considered negligible. Accordingly, the object dimensions(except for the height) may be inferred. This embodiment may save spaceon the system 100.

In yet another embodiment, four measurement sets may be taken withoutusing any reference features or measurement structure. This embodimentmay require utilizing the sensing concept with dimensions (as in FIG.9B). Further, between each measurement set, the end effector 130 isturned by 90 degrees. At the end, the sets are fused to calculate thefinal dimensions (e.g. in a global optimization step) of the object.Furthermore, additional sets could be added to estimate the parametersof a more detailed geometric model of the object (e.g. opposite sidesnot parallel).

The system 100 may further include a structure specifically configuredand having precisely known geometry (or color/brightness patterns ofprecisely known dimensions) that may be perceived by the profile sensors150 on the end effector 130 (in case of patterns, greyscale images wouldbe transmitted from the profile sensors). The data acquired may enablean automatic process to determine the extrinsic calibration of theprofile sensors 150, which is required to compute more abstractproperties (such as the pose of the gripped object in the gripperreference frame) from the combined measurements of multiple sensors withsufficient precision, after reconfiguration of the end-effector 130, andallows recognition of changes over time (or due to the severe event ofan impact). In one embodiment, the intrinsic calibration of camera withlens in case of custom profile sensors may be additionally determined,in order to detect drift from the off-site calibration or factorycalibration. The system 100 may also include an integrated objectcutting apparatus on the mobile base 110. The cutting tool herein may bea spiral saw or a disc saw. In one embodiment, the cutting apparatus maybe a companion device.

According to various embodiments, the system 100 may also include asupport material application apparatus for applying support material toobjects for vertical (or much inclined) installation. The supportmaterial application apparatus may be configured to inject non-rigidsupport material (e.g. a thermoplastic polymer) in spots underneath theobject brought into a desired pose, which is either vertical orinclined. Accordingly, the support material may be injected into the gapbetween the object to be placed and a previously placed object on thesurface, which may be adjacent and/or below. Hence, the support materialmay be injected between the sides of the objects. According to variousembodiments, the support material application apparatus may be at theend effector. According to various embodiments, the support materialapplication apparatus may be coupled to a reconfigurable structure whichis attached to the system 100, for example at the end effector.

According to various embodiments, once the support material becomesrigid, the object may be released from the end-effector such that theweight of the object may be held by the bonding material adhering theobject to the surface as well as the support material that is at thebottom edge of the object. According to various embodiments,cooling/hardening of the support material may be accelerated, when theobject is in the desired pose, by blowing air at it (to help convection)from the end effector or other means.

According to various embodiments, the support material may includematerials (e.g. UV curing glue) which may harden or cure via activatingor at least accelerating by exposing to light of a certain wavelength(e.g. UV light) or with other specific properties from the end effector.Accordingly, if hardening/curing may be activated by exposing to certainconditions, it may be easier to apply the support material along a baseedge of the object after bonding material is applied to the back surfaceof the object. According to various embodiments, the support materialmay be applied in a way such that the support material may besubsequently set back within the gap from the object surface so as toallow grouting of the gap later. According to various embodiments, thesupport material may be applied in a way such that the support materialmay stick out to facilitate later removal, possibly with the help of aremoval tool.

According to various embodiments, the system 100 may further include anabsolute level referencing apparatus. The absolute level referencingapparatus may be coupled to the end effector (and it that case maysubstitute an inclinometer). The absolute level referencing apparatusmay work together with a component placed on the surface or in the room.According to various embodiments, the absolute level referencingapparatus may provide additional information to be taken into accountfor determining the desired/goal pose, in similar fashion to theinclinometer (may be factored in with the information from the one ormore sensor unit 150 to help reduce drift). According to variousembodiments, while the inclinometer may provide an absolute attitude(roll/pitch) reference, the absolute level referencing apparatus mayprovide an absolute level reference (e.g. absolute reference plane inthe room, reference for roll/pitch/z). According to various embodiments,the absolute level referencing apparatus may include projecting a laserplane from the component placed in the room (e.g. a corner). Accordingto various embodiments, if at low level, one or multiple detectors maybe coupled to the end effector. According to various embodiments, thedetectors may also be coupled to the base of the system 100, on thesides (when projection low) or on pole(s) on top (when high).

According to various embodiments, the system 100 may include two roboticarms. Accordingly, in the two robotic arms configuration, the system 100may be able to handle heavier objects which may be too heavy to behandled with one robotic arm. In such an embodiment, the one or moresensors units may sense the object to be placed with respect to a sensorframe. By determining the relative difference between the current poseof the object to be placed and the desired pose of the object to beplaced, the two robotic arms may be controlled accordingly to move theobject to be placed, which is held by the to robotic arms. According tovarious embodiments, the two robotic arms may include two parallelmulti-DOF serial kinematics.

As shown FIG. 12, a bonding material application apparatus 170 for abonding material such as tile adhesive may include a real time controlsystem, a motor driver, a direct current (DC) electrical motor, a ballor trapezoidal screw, a hydraulic transmission having hydrauliccylinders, a metal piston, removable & vertically mounted tube (e.g.plastic), an end plate and an applicator plate. The bonding materialapplication apparatus 170 may be integrated with a robotic unit(described above).

In one embodiment, the real-time control system and motor driver may beprovided as a control unit 171. The real-time control system and motordriver may be connected to the DC electrical motor 172 with an opticalencoder and gearbox.

The DC electrical motor 172 may be further coupled to a ball or atrapezoidal screw extending rod actuator 173 with magnetic switches.

The ball or trapezoidal screw extending rod actuator 173 may be coupledto a hydraulic transmission 174. The hydraulic transmission may includeone or two large, single- or multi-stage (possibly constant speed)single or double acting hydraulic cylinders in parallel.

The hydraulic transmission 174 may be coupled further to a hydrauliccylinder 175 via hydraulic piping 176. The hydraulic cylinder 175 may beconnected to a metal piston. The hydraulic cylinder 175 may include oneor three small double- to quadruple-stage (possibly constant speed)hydraulic cylinders, arranged to be in a triangle (for three cylinders).

In another embodiment, as an alternative to the hydraulic transmission174, the rod actuator 173 could be connected to a scissor mechanism(also in a configuration where both ends of the actuator are attached toscissor members, in which case a hydraulic transmission system withsingle-stage cylinders may still remain).

The metal piston connected to the hydraulic cylinder 175 maybeconfigured to removably receive a (possibly self-lubricating) plasticpiston, which tightly closes the lower end of a removable, verticallymounted (e.g., plastic) tube (also referred to as a cylinder or areservoir) that may form a bucket to store 10-30 liters ofapplication-ready bonding material such as tile adhesive (and hasfeatures which facilitate removal of the removable tube from the bondingmaterial application apparatus 170).

The removable tube that holds adhesive material may be fitted into arigid, vertically installed metal tube 177 configured for supporting theremovable tube, which is closed with a round plastic or metal end plate178 which may have a square grid of holes 179 b for the adhesive to passthrough. The metal plate 178 may be secured to the fixed tube 177 by ascrewed-on ring or a hinge and multiple latches. The metal plate 178 mayinclude brackets screwed on the upper side, which allow insertion of anexchangeable applicator plate 179 a having the square grid of holes 179b. Each hole may include a cavity of specific geometry (such as ahalf-sphere, opening to the upper side of the applicator). Further, theapplicator plate 179 a may possibly have a coating that repels theadhesive and a border around all cavities (possibly with elasticfeatures to support its function as a seal).

The method of applying bonding material using the bonding materialapplication apparatus 170 for a bonding material may include a stepwhereby the underside of an object is pressed against the applicatorplate 179 a. The pressing is performed by means of the robotic arm 120and results in enclosing a cavity formed between the underside of theobject 101 and the applicator plate 179 a).

Subsequently the method includes a step whereby the bonding material ispressed into the enclosed cavity until it is filled.

The method of bonding also includes a step whereby the object 101 islifted up (and if applicable, shifted in the X-Y plane to move anuncovered area over the applicator plate 179 a). According to variousembodiments, the bonding material may be separated from the applicatorplate 179 a when the object 101 is lifted up and thus leaving a layer ofadhesive on the object. According to various embodiments, a separationlayer within the bonding material may be created when the object 101 islifted up, with one part of the bonding material sticking to theapplicator plate 179 a and the other part sticking to the object 101.

Finally, the method also includes a step whereby the object 101 isplaced onto the planar surface (e.g., screed floor), in a slow Z motion,possibly combined with fast motions in the object underside's plane(later followed by closed-loop fine positioning), configured to spreadthe bonding material (while potentially slightly lowering itsviscosity), leading to a uniform layer, covering completely both theplanar surface and the object underside.

In another embodiment, a method of operating the system 100 includesmultiple steps.

The method includes a first step of loading initially known informationrequired by a control program for laying the objects (a “work plan”).The information and the work plan may be loaded into the system 100beforehand (before bringing the system 100 to the worksite). Theinformation or the work plan may also be created in situ ad hoc andloaded into the system 100 at the worksite. For example using directinputs into a graphical user interface (e.g. to set the parameters ofthe object to be placed), defining certain boundaries in the room with areference object that is perceived by the system 100, or moving parts ofthe system 100 to a to be taught location, etc. The information mayinclude (i) dimensions (ideal or measured beforehand), mass, surfaceproperties and arrangement (i.e., a ideal goal/desired pose relative toeach other or absolute, relative to a global frame of reference) of theobjects to be placed, and (ii) shape/dimensions of the planar surface onwhich the objects are to be placed, of adjacent or enclosing planarsurfaces or a full two- or three-dimensional representation of thegeometry (i.e., a “map”) of the surrounding environment.

The method includes a second step of placing one or more physical borderreferences on the planar surface with a pre-defined offset from wherethe border(s) of the first object(s) are desired to be placed. Thephysical border references may be for each dimension in the plannedobject arrangement (e.g., a column) that exceeds a single object. Thephysical border reference may be placed in order to fix the position andorientation of the planned object arrangement on the planar surface. Inone embodiment, when an augmented sensing concept is utilized, a singlemain reference may be sufficient. It should be appreciated, however,that while the objects are placed relative to each other with a highprecision, the position of the whole arrangement with respect to a roomis of lower precision due to the limitation placed from the precision ofthe mobile platform localisation. This may be alleviated by placing suchphysical border references which the system can perceive more precisely.According to various embodiments, the physical border references mayinclude previously placed objects.

In one embodiment, the physical border reference is a wire rope that isstrained at approximately 5 mm to 20 mm height above the planar surface.If needed, the physical border reference is aligned to a laser lineprojected (e.g., by a device available as “tile laser”) from the cornerof the planned object arrangement (where two straight segments of theoutline of the planned object arrangement intersect). According to otherembodiments, the wire rope may be replaced with a fibre rope or tape,which is possibly stored in a self-retracting reel fixed to the ground.

In another embodiment, the physical border reference may be an outerwall/peripheral structure of a frame in which the objects can be placed.

In another embodiment, the border reference is a laser line projected(e.g., by a device available as a “tile laser”) onto the planar surfaceand the end effector 130 is equipped with sensors to perceive it.

In an alternative embodiment, some sides of the outline of the plannedobject arrangement may not be straight (i.e. are composed of sides ofthe objects that are not collinear or parallel), for example in anobject arrangement where every other row is shifted by half of theobject length (e.g. as commonly seen with 300 mm×600 mm tiles).Accordingly, the physical border reference may include preciselymanually laid cut objects with the cut sides aligned with the border ofthe planar surface.

In another embodiment, a tracking total station is placed on the planarsurface, for example at the corner of the area where the objects are tobe placed. In such instance, two or more points on the planar surfacemay be referred in order to fix the position and orientation of theplanned object arrangement on the planar surface.

The method also include a third step of loading the system 100 with thephysical resources required to carry out the construction job (e.g.,laying an arrangement of objects), i.e. loading it with the objects tobe placed (e.g., ceramic tiles) in the next batch. Loading may includedocking a trolley stacking the objects to the system 100. Loading mayalso include placing the physical resources nearby (e.g. at the entranceof a room in case of a cable suspended platform).

The loading may also include loading the system 100 (specifically thebonding material application apparatus 170 and/or the support materialapplication apparatus) with a bonding material (e.g., cement-based tileadhesive) and/or support material.

Furthermore, the loading may also include loading the system 100 withrecharged swappable battery/batteries.

In one embodiment, the third step may be carried out with a dedicatedrefill trolley 162 in FIG. 12, which is configured to facilitatetransporting and unloading (and placing, in some instances) of theobjects onto the system 100 with the aim to simplify and speed up theprocess and minimize the lifting work for the operator.

The refill trolley 162 may furthermore be electrically actuated and ableto both autonomously navigate between the material storage andpreparation location and the system 100.

The method may also include a fourth step of placing the system 100,which has a wheeled or legged mobile base 110, nearby a physicalreference on a surface even enough (e.g., a screed floor) to allowreaching its working positions by using its drive train and starting thecontrol mechanism 180 that progressively places the objects 101 in thesub-area as marked off from the overall area by the physical references(after initially finding and driving close enough to such references).The autonomous function of finding and driving close to such referencesmay be an optional function of the system 100. This may be activatedafter the operator placed the system 100 onto the drivable floor (e.g.screed floor). If the operator places the system directly at the workingposition, this function may not be activated. In case the planar surfaceon which the objects are to be placed is vertical, the planar surface onwhich the system 100 is to be placed may be a different, likelyperpendicular and adjacent surface.

The fourth step of the method may further include manually steering thesystem 100 with a remote controller to position it within the reach ofthe physical reference(s) by the robotic arm 120 (e.g., intersection orcorner of two physical references), i.e. at the starting point.

The system 100 may locate and may reach the starting point as describedautonomously using information from the work plan and its sensors.

The second step is repeated for placing an additional batch of objects(e.g. when the system 100 has run out of objects to place). In additionto that, the second to fourth steps may be repeated when starting toplace additional batches of objects in a new sub-area of the overallarea where the objects are to be placed (likely a disjoint region).

FIG. 14 shows a schematic top view of a system (or an apparatus) 1400for placing objects on a surface according to various embodiments. Thesystem 1400 may include a mobile platform 1410 with four powered steeredcaster wheels (not visible) (e.g. omni drives). The system 1400 may alsoinclude a bonding material application apparatus 1470. The system 1400may also include an object storage apparatus 1460 for stacking objectson the base 1410 (e.g. for flat objects up to 60×30 cm). The system 1400may include a 6 DOF serial kinematic (all revolute joints) robotic arm1420. The robotic arm 1420 may include a non-reconfigurable end effector1430 with four integrated profile sensors 1450 for the basic sensingconcept.

FIG. 15A shows a schematic top view of a system (or an apparatus) 1500for placing objects on a surface according to various embodiments. FIG.15B shows a schematic side view of the system 1500 of FIG. 15A withrobotic arm 1520 extended. The system 1500 may include a mobile platform1510 with four powered steered caster wheels 1512 (e.g. omni drives).The system 1500 may also include a bonding material applicationapparatus 1570. As shown, a separate object stack trolley 1562 forobjects up to 60×60 cm may be docked on a side (e.g. right side) of thesystem 1500. The system 1500 may include a robotic arm 1520 with 5 DOFserial kinematic (1prismatic joint 1521 for Z long, 3 revolute joints1522, 1523, 1524 with parallel vertical axes, “SCARA” configuration, 1prismatic joint 1524 for Z short, technically before last revolute joint1524). The robotic arm 1520 may have a further 3 DOF parallel kinematic(3 prismatic joints 1527, 1528, 1529) for roll/pitch/z fine adjustment(J6-8). In one embodiment, the robotic arm 1520 may not have theprismatic joint 1524 before the last revolute joint. In one embodiment,the robotic arm 1520 may include a non-reconfigurable end effector withfour custom profile sensors for augmented concept including dimensions.FIG. 15B shows a side view of the system 1500 with the manipulator 1520extended without gripping any object, and without the object stack beingvisible.

In one embodiment, the method of operating the system 100 may furtherinclude application on prefabricated prefinished volumetric construction(PPVC) (as shown in FIG. 16) and prefabricated bathroom unit (PBU)facilities where the system 100 can be employed in PPVC or PBU unitscomposed of a steel frame with Engineered Cementitious Composite-Cretewalls or with Autoclaved Aerated Concrete walls or with a combination ofECC-Crete and AAC walls, reinforced concrete (full volumetric concreteconstruction), a metal frame with drywalls, a metal frame with metalwalls, and a metal frame with lightweight panels.

The method may also be applied on construction sites or in automatedproduction processes in facilities (e.g., Integrated Construction andPrefabrication Hubs (ICPHs)) where precast concrete or steel structuresare used or produced or stored. The ability of the system 100 to beemployed directly in the facility or by the ability of the said basejoint 121 of the said robotic arm 120 to be mounted on prefabricatedhybrid structural systems like precast column and steel structures(PCSS) which may use prefabricated reinforcement cages or on slabs whichmay use carpet reinforcement, etc.

The method may also be applied in Cross Laminated Timber or GluedLaminated Timber constructions of walls, floors, roofs or structuralbeams and columns, or in facilities where CLT or Glulam are used orproduced or stored if any part of the construction is to be finishedwith engineered timber flooring having a top layer of hardwood and abottom core of plywood layers in addition to conventional parquet.

The method may be applied in a semi- or fully automatedconstruction/prefabrication facility (as shown in FIG. 17) where thebase joint 121 of the robotic arm 120 is mounted on a gantry 1909 wherea multi-robotic system based on an overhead running gantry system 1900is installed to enable object placement possibly with human-machinecooperation, as well as advanced automated factory-based digitalfabrication processes.

The method may be performed using embodiments of the system 100 (heavyor light weight, collaborative, high or low payload industrial robotarm) with 10 kilograms (kg) or more payload where the operator can worktogether in a shared workspace without safety fences to place objectsfor surface finishing in commercial buildings, hospitals, communitycentres, “hawker” centres aka food courts, airports, MRT or trainstations in addition to public and private housing projects.

The method may also be utilized on a heavyweight non-collaborativehigh-payload industrial robot arm, whereby the operators do not worktogether in a shared workspace without safety fences in order to placeobjects for finishing surfaces in commercial buildings, hospitals,community centres, “hawker” centres aka food courts, airports, MRT ortrain stations in addition to public and private housing projects.

There are other features that may form part of one or more embodimentsthat are mentioned above. These features may include initiallocalization and mapping in a room with a surface where objects are tobe placed (e.g. using two dimensional simultaneous localization andmapping: 2D-SLAM or e.g. using three dimensional simultaneouslocalization and mapping: 3D-SLAM, considering initially only featuresof the room), which may be utilized for comparing the generated map withthe map attached to the work plan and report significant deviations.

The features may also include cleaning of optical sensors by moving themover a compressed air outlet on the platform 110, generating a detailedquality assurance (QA) work report (incl. a 3D model) showing theprecision achieved and highlighting problematic areas that requirecloser inspection and perhaps manual correction.

The features may also include dedicated sensing to ensure a clear spacewithin base footprint and arm workspace during operation, cleaning ofthe underside of objects 101 from dust with compressed air prior toadhesive application for better bonding, cleaning of the floor from dustwith compressed air from the end effector 130 prior to object placement,just after placing an object 101 when the gripper is empty, for betterbonding, determining required dimensions of partial objects such asborder objects (from sensed laid full objects and walls) and submissionto operator, a separate on-site cutting and stacking machine (tray to bemoved onto the system 100) or a cutting apparatus on the system 100itself, optical sensing and analysis of an object's pattern (e.g., fornon-homogeneous tiles) before gripping, with option to store a smallnumber of objects on the system 100 (or sort out), to allow forautomatic matching of patterns within a cohort.

Furthermore, the features also include photography of every laid objectfrom the end effector 130 or gripper and automatic, pose-correctedstitching together to an image of the whole object arrangement, as avisual documentation of the work result, suitable for remote inspection;inspection of floor surface prior to object placement by means ofcomputer vision to ensure no debris is present; using a camera on theend effector 130 for recognition of QR codes (or similar identifyingmachine-readable optical label) previously placed in the room by theoperator as additional information to identify the room and to referenceand possibly download a corresponding work plan and/or BIM model; usinga camera on the end effector 130 (possibly part of a custom profilesensor) for recognition and localisation (6DOF pose, in the end effectorreference frame) of artificial visual landmarks of known dimensions (akafiducials, usually pattern of high contrast and providing richinformation, usually providing unique identification, e.g. QR, AprilTagsor ArUco markers), placed in advance by the operator (e.g. fiducialstickers, freely placed on the wall every meter at about 40 cm height,or a tape with printed-on, possibly equally spaced at known distance,fiducials applied to the wall), with pose measurements used in a similarfashion as those of the objects placed (represented in the model, fusedin a similar fashion with earlier information as described before,serving as a map), allowing to reduce drift, particularly duringrelocation of the platform.

In one embodiment, a control mechanism 180 for operating the system 100may include steps to execute a high-level cycle to place an object 101,which includes (i) parse a work plan, (ii) localize the system 100,(iii) find physical references and starting point, possibly measuringthe surface where the first object is to be placed (iv) lower the feet114 of the system 100 to render it stationary, (v) pick up an object101, measure its pose on the gripper/end effector 130, (vi) measure itsdimensions, (vii) apply the bonding material to the object's undersideand possibly the planar surface, (viii) place the object 101 relative tothe physical references on the planar surface, (ix) measure theresulting object 101 pose after release, and possibly the surface wherethe next object is to be placed, (x) pick up next object and repeatsteps until no more objects are left in the patch, (xi) retract the feet114, (xii) drive the base 110 to a next placement position, (xiii)measure the pose of a previous object to reduce uncertainty in the basepose, (xiv) lower the feet 114 and repeat the steps (v)-(x) mentionedabove.

The control mechanism 180 may also include a step which implements aspecific sensing and control concept necessary to achieve the requiredplacement precision. This can be performed through (i) continuousmeasurement of the 6 DOF pose (or 3 DOF of the pose if it is otherwisemechanically restricted in a precisely known fashion) of the object tobe placed 101 with respect to a frame of reference on the end effector,(ii) closed-loop control of the 6 DOF pose of the end effector and byextension the object to be placed 101 relative to the previously placedadjacent objects on the surface (and/or the floor plane and/or physicalreferences such as a string placed by the operator) using real-timesensed or estimated information to determine such pose, (iii)determining a goal pose for the object to be placed depending onadjacent and further away objects, using information on such objectsfrom the model, optimizing towards geometric, visual criteria on a localand global scale, (iv) possibly estimating the poses (and potentiallytheir full history) of the end effector and all objects placed on thesurface using probabilistic multi-modal sensor fusion, realized with arecursive filter or a global optimization scheme (with potentiallyfurther variables incorporated into the state such as poses of otherparts of the robot or features of the room), building up and updating amodel of work done, and (v) an alternative force-torque-sensing basedpositioning for gap free arrangements.

The control mechanism 180 may also include a user interface to thesystem 100 that provides multiple features. Amongst the features are:(i) detachable corded tablet (similar to teach pendant) displaying atouch-GUI, (ii) basic operation of: load, preview and start execution ofplans, (iii) in case of severe placement errors, that the operator ispresented with choices on how to correct, (iv) means to send alerts tooperator's smartphone, (v) an interface to correct object layout in caseplan not executable, auto-generated suggestions for alternatives and(vi) an interface to adjust parameters of the plan, e.g. if differentobjects need to be used on-site which are heavier than the ones the planwas made for.

In one embodiment, a desktop application or tablet software to generateand assemble the initially known information (the “work plan”) requiredby the control mechanism 180 of the system 100 enables the followingworkflow: (a) import floor plan (or plan of the planar surface) fromvarious formats (e.g. from BIM), (b) select the relevant area forobjects to be placed, (c) specify the types of objects to be used in thelayout (e.g. shape, ideal dimensions, weight etc.; possibly selectobjects from database), (d) compose a “primitive stamp” for the desiredpattern, incl. desired gap size (its outline is checked for feasibilityof execution, e.g. toothing of elements), (e) set the origin of thecoordinate frame from which the stamp is used to expand the pattern overthe placement area, shift until satisfied with the segmentation on theborders, (f) for non-homogeneous objects which have been digitallyinventoried and for which the database contains adjacency relationshipsand possibly an image of the upper side, link at least one of thegeneric objects of the expanded pattern to a uniquely identified objectfrom the database, mapping the object motifs onto the object layout, (g)make manual adjustments on a per-object basis, (h) generate the machineplan for the uncut objects (and save the plan semantics in a projectfile), and (i) automatic decomposition of the object layout into asequence of ordered subsets of objects referred to as patches (so thatall objects in a patch can be reached from the same stationary basepose; i.e. based on the reachability of the arm, the set of patch typesthat results in for the chosen pattern, and the manoeuvrability of themobile platform) and (j) automatic planning of the associated stationarymobile base poses and possibly the paths between them.

In a first embodiment, a robotic system for placing objects on a planarsurface comprising: a base assembly that is capable of moving in aplanar direction; a robotic arm having two ends, wherein one end iscoupled to the platform; an end effector having two surfaces, whereinone of the surfaces is coupled to the other end of the robotic arm; oneor more or a plurality of suction cups disposed on another surface ofthe end effector, wherein the suction cups are coupled to a vacuumsource and the plurality of the suction cups are capable of gripping anobject when the vacuum is activated; and a plurality of profile sensorsdisposed on sides of the end effector, wherein the profile sensors sensethe object and a previously placed object.

In one embodiment, the robotic system described in the first embodimentincludes at least two profile sensors. For example the robotic systemmay include three profile sensors or four profile sensors or more.

In one embodiment, the robotic system described in the first embodimentwherein a resulting gap between the object and a previously placedobject is a first distance, and wherein differences between the firstdistance and an expected first distance is 0.5 millimetres or less.

In one embodiment, the robotic system as described in the firstembodiment wherein the expected first distance is between a range of 0.5millimetres to 12 millimetres, or 1 millimetre to 5 millimetres.

In one embodiment, the robotic system as described in the firstembodiment wherein the object is a tile.

In one embodiment, the robotic system as described in the firstembodiment further comprising: an air compressor that is coupled tovacuum ejector to enable suction effect through the plurality of suctioncups; an object storage apparatus; and a bonding material storage andapplication system. The object storage apparatus may be detachable.

In one embodiment, the robotic system as described in the firstembodiment is controlled using a closed-loop control mechanism, whereininformation is received from the plurality of profile sensors and usedfor controlling the pose of the end effector.

In one embodiment, the robotic system as described in the firstembodiment wherein the end effector further comprises: a plurality ofLiDAR sensors; an orientation reference measurement system; a vacuumejector; a precision rabbet with calibrated geometry utilized formeasuring dimensions of the object; and a plurality of features thatprovide a solid contact surface to the upper side of the object forconstraining its movement in direction of the end joint.

In one embodiment, wherein the end effector of the robotic system asdescribed in the first embodiment is configured based on a selectedconcept from a plurality of concepts consisting of: basic sensingconcept, an augmented sensing concept and any combination thereof

In one embodiment, the basic sensing concept further comprisescontinuous measurement of all degrees of freedom of the currentlygripped object and one degrees of freedom less from all degrees offreedom of the previously placed objects.

In one embodiment, the augmented sensing concept further comprisescontinuous measurement of all degrees of freedom of the currentlygripped object and all degrees of freedom of the previously placedobjects by sampling an additional side (adjacent to the side sampled inthe basic concept) of the previously placed objects, thus reaching fulldegrees of freedoms of the previously placed objects.

In one embodiment, the basic sensing concept and the augmented sensingconcept further comprise measurement of dimensions of the object bysampling at least one additional side of the object which is opposite toa side that is to be aligned.

In a second embodiment, a method of operating a robotic unit that placesobjects on a planar surface comprises: finding a physical reference anda starting point; using a gripper on the robotic unit, picking up theobject; using a plurality of profile sensors mounted on the roboticunit, measuring poses and dimensions of the object on the gripper;placing the object relative to physical references or previous objectson a planar surface; and using the plurality of profile sensors,measuring pose of the object after releasing. The method may furtherinclude applying a bonding material to the object. The method mayfurther include measuring the planar surface where the next object is tobe placed when a bonding material is used.

In one embodiment, the method as described in the second embodimentfurther comprises: using the gripper on the robotic unit, picking upanother object; using the plurality of profile sensors on the roboticunit, measuring poses and dimensions of the another object on thegripper; placing another object relative to the physical references onthe planar surface; and using the plurality of profile sensors,measuring poses of another object after releasing.

In one embodiment, the method described in the second embodiment furthercomprising: continuously measuring poses of the object on the gripper;using a closed-loop or a feedback control mechanism within the roboticunit, controlling the pose of the gripped object relative to previouslyplaced objects.

In one embodiment, the method described in the second embodiment furthercomprising: estimating the pose of the gripper and all placed objectsusing probabilistic multi-modal sensor fusion; and building up a modelof the placed objects in the process.

In a third embodiment, a method of operating a robotic unit comprising:using a control program, receiving an initially known informationrequired for placing the objects; placing one or more physical borderreferences on a planar surface with a pre-defined offset from where theborder of the first objects are desired; loading the robotic unit withthe first objects; and placing the robotic unit relatively near to thearea of work.

In one embodiment, wherein the initially known information of the methoddescribed in the third embodiment is selected from a group ofinformation consisting of: dimensions, mass, surface properties andarrangement of the first objects.

In one embodiment, wherein the initially known information of the methoddescribed in the third embodiment also comprises shape and dimensions ofthe planar surface on which the first objects are to be placed.

In one embodiment, the method of operating the robotic unit as describedin the third embodiment may further comprises: using the controlprogram, receiving an initially known information required for layingsecond objects; placing one or more physical border references on theplanar surface with a pre-defined offset from where the border of thesecond objects are desired; loading the robotic unit with the secondobjects; and placing the robotic unit relatively near to the area ofwork.

According to various embodiments, there is provided a system for placingobjects on a surface. The system may include a base, a robotic armcoupled, at an end thereof, to the base, an end effector coupled to theother end of the robotic arm. The end effector may be configured forreleaseably coupling to an object to be placed on the surface. Thesystem may further include one or more sensor units on a sensor frame.The one or more sensor units may be configured for sensing atwo-dimensional profile data including at least two two-dimensionalprofiles together comprising at least three boundary portions of theobject to be placed and at least boundary portions of objects on thesurface. At least two of the three boundary portions of the object to beplaced may be from substantially non-parallel sides. At least two of thethree boundary portions of the objects on the surface may be fromsubstantially non-parallel sides. The system may further include aprocessor configured to determine at least three degrees of freedom ofthe object to be placed with respect to the sensor frame and six degreesof freedom of the sensor frame with respect to the objects on thesurface in a three-dimensional space for determining a current pose ofthe object to be placed with respect to the objects on the surface basedon the two-dimensional profile data. Further, the system may beconfigured to place the object based on differences between the currentpose and a desired pose of the object to be placed determined from amodel of objects on the surface in the three-dimensional space.

According to various embodiments, the one or more sensor units mayinclude at least two profile sensors.

According to various embodiments, the one or more sensor units mayinclude one or more imaging devices and one or more light emittingunits. The one or more light emitting units may be configured to projecta single line or multiple lines or a predetermined pattern of lines.

According to various embodiments, the one or more sensor units may bemounted to an expandable frame structure coupled to the end effector.

According to various embodiments, the one or more sensor units may bemounted to a fixed-sized frame structure coupled to the end effector.

According to various embodiments, the robotic arm may be configured toplace the object with a resulting gap between the placed object and theobject on the surface. The resulting gap may be a first distance. Thedifferences between the first distance and an expected first distancemay be 0.5 millimetres or less, or 0.4 millimetres or less, or 0.3millimetres or less, or 0.2 millimetres or less.

According to various embodiments, the expected first distance may bebetween a range of 0.5 millimetre to 12 millimetres.

According to various embodiments, the object may be a tile.

According to various embodiments, the system may further include one ormore suction cups disposed at the end effector, and a vacuum generatorcoupled to the one or more suction cups. The vacuum generator may beconfigured to enable suction effect through the one or more suctioncups. The vacuum generator may include an air compressor coupled to avacuum ejector.

According to various embodiments, the base may include wheels or legs.

According to various embodiments, the base may be suspended above thesurface by cables or overhead rails.

According to various embodiments, the robotic arm may include acomposite joint having at least three parallel prismatic joints. Each ofthe three parallel prismatic joints may include a revolute joint oruniversal joint at an end, and a spherical joint at the other end. Theend effector may be coupled to the composite joint.

According to various embodiments, the system may further include anobject storage apparatus. According to various embodiments, the objectstorage apparatus may be configured to be detachable from the system.

According to various embodiments, the system may further include abonding material application apparatus.

According to various embodiments, the system may further include asupport material application apparatus.

According to various embodiments, the processor may be configured tocontrol a pose of the end effector via a closed-loop control mechanismbased on information received from the one or more sensor units.

According to various embodiments, the end effector may further includeat least one of a LiDAR sensor, or an orientation reference measurementapparatus, or a force-torque sensor, or a vibrator, or a precisionrabbet with calibrated geometry utilized for measuring dimensions of theobject, or a structure that provides a solid contact surface to theupper side of the object for constraining its movement in normaldirection to the upper side.

According to various embodiments, the system may include two roboticarms coupled to the base.

According to various embodiments, the system may further include anauxiliary camera.

According to various embodiments, the system may further include anobject cutting apparatus.

According to various embodiments, there is provided a method for placingobjects on a surface. The method may include providing a system. Thesystem may include a base, a robotic arm coupled, at an end thereof, tothe base, an end effector coupled to the other end of the robotic arm.The end effector may be configured for releaseably coupling to an objectto be placed on the surface. The system may further include one or moresensor units on a sensor frame. The one or more sensor units may beconfigured for sensing a two-dimensional profile data including at leasttwo two-dimensional profiles together comprising at least three boundaryportions of the object to be placed and at least three boundary portionsof objects on the surface. At least two of the three boundary portionsof the object to be placed may be from substantially non-parallel sides.At least two of the three boundary portions of the objects on thesurface may be from substantially non-parallel sides. The system mayfurther include a processor configured to determine at least threedegrees of freedom of the object to be placed with respect to the sensorframe and six degrees of freedom of the sensor frame with respect to theobjects on the surface in a three-dimensional space for determining acurrent pose of the object to be placed with respect to the objects onthe surface based on the two-dimensional profile data. The method mayfurther include placing, using the system, the object based ondifferences between the current pose and a desired pose of the object tobe placed determined from a model of objects on the surface in thethree-dimensional space.

According to various embodiments, the method may further include pickingup the object to be placed using the end effector of the robotic arm.The method may further include measuring a pose of the object on the endeffector using the one or more sensor units. The method may furtherinclude placing the object relative to object on the surface. The methodmay further include measuring the pose of the placed object using theone or more sensor units.

According to various embodiments, the method may further include pickingup a further object using the end effector of the robotic unit. Themethod may further include measuring a pose of the further object on theend effector using the one or more sensor units. The method may furtherinclude placing the further object relative to one or more objects onthe surface. The method may further include measuring the pose of theplaced and released further object using the one or more sensor units.

According to various embodiments, the method may further includemeasuring dimensions of the object to be placed using the one or moresensor units.

According to various embodiments, the method may further includemeasuring the surface where the next object is to be placed using theone or more sensor units.

According to various embodiments, the method may further includecontinuously measuring the pose of the object on the end effector. Themethod may further include controlling the pose of the object on the endeffector relative to the one or more objects on the surface based on thecontinuously measured poses of the object on the end effector.

According to various embodiments, the method may further includebuilding up the model of the placed objects. The method may furtherinclude determining a pose for the object to be placed based on themodel of the placed objects.

According to various embodiments, the method may further include placingone or more physical border references on the surface with a pre-definedoffset from where the sides of the objects to be placed are desired. Themethod may further include loading the system with the objects. Themethod may further include and placing the system in a work area.

According to various embodiments, the method may further includeapplying bonding material. The method may include applying bondingmaterial on a back surface of the object to be placed. The method mayinclude applying bonding material on the surface on which the object isto be placed.

According to various embodiments, the method may further includeapplying support material on at least one side of the object to beplaced.

According to various embodiments, the method may further includereceiving a predetermined information for placing a first set of theobjects.

According to various embodiments, the method may further include thepredetermined information may be selected from a group of informationconsisting of: dimensions, mass, surface properties, ideal desired posesor arrangement of the first set of the objects and properties of one ormore physical border references.

According to various embodiments, the predetermined information mayinclude shape and dimensions of the surface on which the first set ofobjects is to be placed.

According to various embodiments, the method may further includereceiving a predetermined information required for placing a second setof objects. The method may further include placing one or more physicalborder references on the surface with a pre-defined offset from wherethe sides of the second set of the objects to be placed are desired. Themethod may further include loading the system with the second objects.The method may further include placing the system in the work area.

Various embodiments have provided a system having sensing capabilitieson an end effector that would enable measuring sufficiently many degreesof freedom of the involved objects (both gripped and placed previously)to achieve an accurate placement in the presence of a bonding material(which can lead to small changes in the poses of objects shortly afterplacement which need to be preceived and taken into account) or activelycompensate for the movement of the gripped object during placement (dueto forces from interactions with the bonding material). Further, thesesensing capabilities may also enable placement of objects so as tofollow a curved surface (normal vector of upper surface of an objectapproximately in parallel with normal vector of surface underneath).

Various embodiments have also provided a system that recognizes the needto control the application of the bonding material to (a) have morecontrolled interactions of it with the object to be placed and/or thesurface (e.g. of force experienced by manipulator during placement) andto (b) avoid an inadequate uncontrolled application negatively affectthe sensing during placement and the bonding and support of the objectslong after placement and to (c) potentially adjust for unevenness in theplanar surface and to (d) in general ensure the object's weight isadequately supported after its release so its pose remains stable, allin order to achieve a robust process. Further, this may allow achievinga slightly inclined surface formed by the upper surfaces of the objectswith respect to the surface the objects are placed on.

Various embodiments have also provided a system that recognizes the needfor closed-loop control of the pose (position and orientation in3-dimensional space) of the object to be placed, with respect to a fixedreference frame (e.g. on the surface), in order to eliminate theinfluence of kinematic errors from the robotic arm, and/or the need ofvarious additional measures such as model-based active compensationagainst the accumulation of local errors to critical global errors, andimplemented such closed-loop control and modelling for activecompensation in the system.

Various embodiments have provided a system that combines a low rate“visual” position control loop with high-bandwidth joint level controlto achieve good control behaviour, which is considered superior than animage-based or a position based visual servoing without use ofmanipulator joint feedback.

While the invention has been particularly shown and described withreference to specific embodiments, it should be understood by thoseskilled in the art that various changes, modification, variation in formand detail may be made therein without departing from the scope of theinvention as defined by the appended claims. The scope of the inventionis thus indicated by the appended claims and all changes which comewithin the meaning and range of equivalency of the claims are thereforeintended to be embraced.

1. A system for placing objects on a surface comprising: a base; arobotic arm coupled, at an end thereof, to the base; an end effectorcoupled to the other end of the robotic arm, wherein the end effector isconfigured for releaseably coupling to an object to be placed on thesurface; one or more sensor units on a sensor frame, wherein the one ormore sensor units is configured for sensing a two-dimensional profiledata including at least two two-dimensional profiles together comprisingat least three boundary portions of the object to be placed and at leastthree boundary portions of objects on the surface, wherein at least twoof the three boundary portions of the object to be placed are fromsubstantially non-parallel sides, and wherein at least two of the threeboundary portions of the objects on the surface are from substantiallynon-parallel sides; and a processor configured to determine at leastthree degrees of freedom of the object to be placed with respect to thesensor frame and six degrees of freedom of the sensor frame with respectto the objects on the surface in a three-dimensional space fordetermining a current pose of the object to be placed with respect tothe objects on the surface based on the two-dimensional profile data,wherein the system is configured to place the object based ondifferences between the current pose and a desired pose of the object tobe placed determined from a model of objects on the surface in thethree-dimensional space.
 2. The system as claimed in claim 1, whereinthe one or more sensor units comprises at least two profile sensors orwherein the one or more sensor units comprises one or more imagingdevices and one or more light emitting units.
 3. (canceled)
 4. Thesystem as claimed in claim 2, wherein the one or more light emittingunits are configured to project a single line or multiple lines or apredetermined pattern of lines.
 5. The system as claimed in claim 1,wherein the one or more sensor units are mounted to an expandable framestructure coupled to the end effector or wherein the one or more sensorunits are mounted to a fixed-sized frame structure coupled to the endeffector.
 6. (canceled)
 7. The system as claimed in claim 1, wherein therobotic arm is configured to place the object with a resulting gapbetween the placed object and the object on the surface, wherein theresulting gap is a first distance, and wherein differences between thefirst distance and an expected first distance is 0.5 millimeters orless, and preferably the expected first distance is between a range of0.5 millimeter to 12 millimeters.
 8. (canceled)
 9. The system as claimedin claim 1, wherein the object is a tile.
 10. The system as claimed inclaim 1, further comprising: one or more suction cups disposed at theend effector; a vacuum generator coupled to the one or more suctioncups, wherein the vacuum generator is configured to enable suctioneffect through the one or more suction cups.
 11. The system as claimedin claim 1, wherein the base comprises wheels or legs or wherein thebase is suspended above the surface by cables or overhead rails; and/orwherein the robotic arm comprises a composite joint having at leastthree parallel prismatic joints, wherein each of the three parallelprismatic joints comprises a revolute joint or universal joint at an endand a spherical joint at the other end, and wherein the end effector iscoupled to the composite joint.
 12. (canceled)
 13. (canceled)
 14. Thesystem as claimed in claim 1, further comprising an object storageapparatus and preferably wherein the object storage apparatus isconfigured to be detachable from the system.
 15. (canceled)
 16. Thesystem as claimed in claim 1, further comprising a bonding materialapplication apparatus and/or further comprising a support materialapplication apparatus.
 17. (canceled)
 18. The system as claimed in claim1, wherein the processor is configured to control a pose of the endeffector via a closed-loop control mechanism based on informationreceived from the one or more sensor units and/or wherein the endeffector further comprises at least one of a LiDAR sensor, or anorientation reference measurement apparatus, or a force-torque sensor,or a vibrator, or a precision rabbet with calibrated geometry utilizedfor measuring dimensions of the object, or a structure that provides asolid contact surface to the upper side of the object for constrainingits movement in normal direction to the upper side.
 19. (canceled) 20.The system as claimed in claim 1, wherein the system comprises tworobotic arms coupled to the base and/or further comprising an auxiliarycamera, and/or further comprising an object cutting apparatus. 21.(canceled)
 22. (canceled)
 23. A method for placing objects on a surface,the method comprising: providing a system comprising a base, a roboticarm coupled, at end thereof, to the base, an end effector coupled to theother end of the robotic arm, wherein the end effector is configured forreleaseably coupling to an object to be placed on the surface, one ormore sensor units on a sensor frame, wherein the one or more sensorunits is configured for sensing a two-dimensional profile data includingat least two two-dimensional profiles together comprising at least threeboundary portions of the object to be placed and at least three boundaryportions of objects on the surface, wherein at least two of the threeboundary portions of the object to be placed are from substantiallynon-parallel sides, and wherein at least two of the three boundaryportions of the objects on the surface are from substantiallynon-parallel sides, and a processor configured to determine at leastthree degrees of freedom of the object to be placed with respect to thesensor frame and six degrees of freedom of the sensor frame with respectto the objects on the surface in a three-dimensional space fordetermining a current pose of the object to be placed with respect tothe objects on the surface based on the two-dimensional profile data;and placing, using the system, the object based on differences betweenthe current pose and a desired pose of the object to be placeddetermined from a model of objects on the surface in thethree-dimensional space.
 24. The method as claimed in 23, furthercomprising picking up the object to be placed using the end effector ofthe robotic arm; measuring a pose of the object on the end effectorusing the one or more sensor units; placing the object relative toobject on the surface; measuring the pose of the placed object using theone or more sensor units; and/or picking up a further object using theend effector of the robotic unit; measuring a pose of the further objecton the end effector using the one or more sensor units; placing thefurther object relative to one or more objects on the surface; andmeasuring the pose of the placed and released further object using theone or more sensor units.
 25. (canceled)
 26. The method as claimed inclaim 23, further comprising measuring dimensions of the object to beplaced using the one or more sensor units and/or further comprisingmeasuring the surface where the next object is to be placed using theone or more sensor units.
 27. (canceled)
 28. The method as claimed inclaim 23, further comprising: continuously measuring the pose of theobject on the end effector; controlling the pose of the object on theend effector relative to the one or more objects on the surface based onthe continuously measured poses of the object on the end effector. 29.The method as claimed in claim 23, further comprising: building up themodel of the placed objects; and determining a pose for the object to beplaced based on the model of the placed objects; and/or placing one ormore physical border references on the surface with a pre-defined offsetfrom where the sides of the objects to be placed are desired; loadingthe system with the objects; and placing the system in a work area. 30.(canceled)
 31. The method as claimed in claim 23, further comprisingapplying bonding material and/or further comprising applying supportmaterial on at least one side of the object to be placed and/or furthercomprising receiving a predetermined information for placing a first setof the objects.
 32. (canceled)
 33. (canceled)
 34. The method as claimedin claim 31, wherein the predetermined information is selected from agroup of information consisting of: dimensions, mass, surfaceproperties, ideal desired poses or arrangement of the first set of theobjects and properties of one or more physical border references,preferably wherein the predetermined information comprises shape anddimensions of the surface on which the first set of objects is to beplaced.
 35. (canceled)
 36. The method as claimed in claim 23, furthercomprising: receiving a predetermined information required for placing asecond set of objects; placing one or more physical border references onthe surface with a pre-defined offset from where the sides of the secondset of the objects to be placed are desired; loading the system with thesecond objects; and placing the system in the work area.